2023-07-29T13:06:59.188 INFO:root:teuthology version: 0.0.1.dev172+g6114016 2023-07-29T13:06:59.189 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --verbose --archive /home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625 --name yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} -- /home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625/orig.config.yaml 2023-07-29T13:06:59.226 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-29T13:06:59.317 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625 branch: reef-release description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} email: yweinste@redhat.com first_in_suite: false job_id: '7356625' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef-release ceph: conf: client: debug monc: 20 debug ms: 1 debug objecter: 20 debug rados: 20 global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 ms type: async osd heartbeat grace: 80 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osd crush smoke test: false mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 mon warn on pool no app: false paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 1 debug bluestore: 1 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd class default list: '*' osd class load list: '*' osd client watch timeout: 120 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd fast shutdown: false osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - reached quota - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(SLOW_OPS\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(OBJECT_MISPLACED\) - slow request - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(SLOW_OPS\) - slow request sha1: 0484097b0572e96f42cf6402cbe0ac8dcb046577 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: debuginfo: true flavor: default sha1: 0484097b0572e96f42cf6402cbe0ac8dcb046577 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef-release sha1: 0484097b0572e96f42cf6402cbe0ac8dcb046577 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 5363 sha1: 0484097b0572e96f42cf6402cbe0ac8dcb046577 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: reef-release suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 0484097b0572e96f42cf6402cbe0ac8dcb046577 targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMq9L1pCZlICkqLKsCQ1cbsTIVQukPVXnoChMPUjBzoDV3riqmHBm32FQWVTGmE7j0ZBfq5L6k7As7l0oYgfcqE= smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRiuvrnGNaq6R3f1uQMQUyTEeSjQdZBYrRH8yP1hs8qaTIr5bJ8hekyG39aJbWdoGjv81vqMy/3vTr8gC0vGJA= tasks: - install: null - ceph: null - workunit: clients: client.0: - rados/test.sh env: ALLOW_TIMEOUTS: '1' timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 61140164d2579c30ffff3970615e0f82e40c6470 timestamp: 2023-07-28_23:11:59 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-29T13:06:59.317 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa; will attempt to use it 2023-07-29T13:06:59.318 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks 2023-07-29T13:06:59.319 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-29T13:06:59.322 INFO:teuthology.task.internal:Checking packages... 2023-07-29T13:06:59.352 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '0484097b0572e96f42cf6402cbe0ac8dcb046577' 2023-07-29T13:06:59.352 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-29T13:06:59.352 INFO:teuthology.packaging:ref: None 2023-07-29T13:06:59.353 INFO:teuthology.packaging:tag: None 2023-07-29T13:06:59.353 INFO:teuthology.packaging:branch: reef-release 2023-07-29T13:06:59.353 INFO:teuthology.packaging:sha1: 0484097b0572e96f42cf6402cbe0ac8dcb046577 2023-07-29T13:06:59.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef-release 2023-07-29T13:06:59.578 INFO:teuthology.task.internal:Found packages for ceph version 18.1.3-11.g0484097b 2023-07-29T13:06:59.579 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-29T13:06:59.585 INFO:teuthology.task.internal:no buildpackages task found 2023-07-29T13:06:59.585 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-29T13:06:59.607 INFO:teuthology.task.internal:Saving configuration 2023-07-29T13:06:59.621 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-29T13:06:59.627 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-29T13:06:59.654 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-29 13:02:09.641667', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMq9L1pCZlICkqLKsCQ1cbsTIVQukPVXnoChMPUjBzoDV3riqmHBm32FQWVTGmE7j0ZBfq5L6k7As7l0oYgfcqE='} 2023-07-29T13:06:59.690 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-29 13:02:09.643293', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRiuvrnGNaq6R3f1uQMQUyTEeSjQdZBYrRH8yP1hs8qaTIr5bJ8hekyG39aJbWdoGjv81vqMy/3vTr8gC0vGJA='} 2023-07-29T13:06:59.690 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-29T13:06:59.699 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-07-29T13:06:59.699 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-07-29T13:06:59.700 INFO:teuthology.run_tasks:Running task console_log... 2023-07-29T13:06:59.782 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1941222e50>, signals=[15]) 2023-07-29T13:06:59.784 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-29T13:06:59.790 INFO:teuthology.task.internal:Opening connections... 2023-07-29T13:06:59.791 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2023-07-29T13:06:59.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-29T13:06:59.890 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2023-07-29T13:06:59.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-29T13:06:59.986 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-29T13:06:59.995 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2023-07-29T13:07:00.013 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2023-07-29T13:07:00.014 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2023-07-29T13:07:00.071 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2023-07-29T13:07:00.071 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2023-07-29T13:07:00.072 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2023-07-29T13:07:00.072 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2023-07-29T13:07:00.072 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2023-07-29T13:07:00.072 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2023-07-29T13:07:00.073 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-29T13:07:00.073 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2023-07-29T13:07:00.073 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2023-07-29T13:07:00.074 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-29T13:07:00.074 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2023-07-29T13:07:00.074 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-29T13:07:00.074 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-29T13:07:00.075 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-29T13:07:00.075 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2023-07-29T13:07:00.103 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2023-07-29T13:07:00.121 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2023-07-29T13:07:00.121 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2023-07-29T13:07:00.177 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2023-07-29T13:07:00.177 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2023-07-29T13:07:00.177 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2023-07-29T13:07:00.178 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2023-07-29T13:07:00.178 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2023-07-29T13:07:00.178 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2023-07-29T13:07:00.178 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-29T13:07:00.178 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2023-07-29T13:07:00.178 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2023-07-29T13:07:00.178 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-29T13:07:00.179 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2023-07-29T13:07:00.179 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-29T13:07:00.179 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-29T13:07:00.179 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-29T13:07:00.179 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2023-07-29T13:07:00.203 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-29T13:07:00.215 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-29T13:07:00.227 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-29T13:07:00.228 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2023-07-29T13:07:00.230 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2023-07-29T13:07:00.246 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-29T13:07:00.254 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-29T13:07:00.255 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2023-07-29T13:07:00.287 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2023-07-29T13:07:00.406 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-29T13:07:00.467 INFO:teuthology.run_tasks:Running task kernel... 2023-07-29T13:07:00.485 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-07-29T13:07:00.485 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-07-29T13:07:00.486 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-29T13:07:00.487 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-29T13:07:00.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-29T13:07:00.503 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2023-07-29T13:07:00.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-29T13:07:00.504 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2023-07-29T13:07:00.559 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-239.el9.x86_64 2023-07-29T13:07:00.560 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-239.el9.x86_64 2023-07-29T13:07:00.560 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2023-07-29T13:07:00.561 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-239.el9.x86_64 2023-07-29T13:07:00.561 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.14.0-239.el9.x86_64 2023-07-29T13:07:00.561 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2023-07-29T13:07:02.713 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-07-29T13:07:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - AppStream 8.8 MB/s | 6.3 MB 00:00 2023-07-29T13:07:04.389 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2023-07-29T13:07:04.538 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2023-07-29T13:07:05.656 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 7.2 MB 00:00 2023-07-29T13:07:06.546 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 7.2 MB 00:01 2023-07-29T13:07:08.634 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 17 MB 00:01 2023-07-29T13:07:08.864 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 14 MB/s | 17 MB 00:01 2023-07-29T13:07:12.616 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 13 kB 00:00 2023-07-29T13:07:12.856 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 13 kB 00:00 2023-07-29T13:07:13.700 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 19 MB/s | 18 MB 00:00 2023-07-29T13:07:14.383 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 13 MB/s | 18 MB 00:01 2023-07-29T13:07:19.856 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-07-29T13:07:19.983 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:07:20.001 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:07:20.001 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-29T13:07:20.002 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:07:20.002 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:07:20.002 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-344.el9 CentOS-BaseOS 4.8 M 2023-07-29T13:07:20.002 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-29T13:07:20.002 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-core x86_64 5.14.0-344.el9 CentOS-BaseOS 19 M 2023-07-29T13:07:20.002 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules x86_64 5.14.0-344.el9 CentOS-BaseOS 37 M 2023-07-29T13:07:20.002 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-core x86_64 5.14.0-344.el9 CentOS-BaseOS 31 M 2023-07-29T13:07:20.003 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:07:20.003 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:07:20.003 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:07:20.003 INFO:teuthology.orchestra.run.smithi040.stdout:Install 4 Packages 2023-07-29T13:07:20.003 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:07:20.004 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 93 M 2023-07-29T13:07:20.004 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 120 M 2023-07-29T13:07:20.004 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:07:20.468 INFO:teuthology.orchestra.run.smithi040.stdout:(1/4): kernel-5.14.0-344.el9.x86_64.rpm 10 MB/s | 4.8 MB 00:00 2023-07-29T13:07:20.672 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-07-29T13:07:20.762 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:07:20.775 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:07:20.775 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-07-29T13:07:20.775 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:07:20.775 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:07:20.776 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-344.el9 CentOS-BaseOS 4.8 M 2023-07-29T13:07:20.776 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-07-29T13:07:20.776 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-344.el9 CentOS-BaseOS 19 M 2023-07-29T13:07:20.776 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-344.el9 CentOS-BaseOS 37 M 2023-07-29T13:07:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-344.el9 CentOS-BaseOS 31 M 2023-07-29T13:07:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:07:20.777 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:07:20.777 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:07:20.778 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2023-07-29T13:07:20.778 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:07:20.778 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 93 M 2023-07-29T13:07:20.778 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 120 M 2023-07-29T13:07:20.779 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:07:21.218 INFO:teuthology.orchestra.run.smithi040.stdout:(2/4): kernel-core-5.14.0-344.el9.x86_64.rpm 16 MB/s | 19 MB 00:01 2023-07-29T13:07:21.643 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-344.el9.x86_64.rpm 5.6 MB/s | 4.8 MB 00:00 2023-07-29T13:07:22.744 INFO:teuthology.orchestra.run.smithi040.stdout:(3/4): kernel-modules-core-5.14.0-344.el9.x86_6 14 MB/s | 31 MB 00:02 2023-07-29T13:07:22.859 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-344.el9.x86_64.rpm 9.3 MB/s | 19 MB 00:02 2023-07-29T13:07:24.701 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-5.14.0-344.el9.x86_64.rpm 9.4 MB/s | 37 MB 00:03 2023-07-29T13:07:25.193 INFO:teuthology.orchestra.run.smithi040.stdout:(4/4): kernel-modules-5.14.0-344.el9.x86_64.rpm 7.1 MB/s | 37 MB 00:05 2023-07-29T13:07:25.196 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:07:25.197 INFO:teuthology.orchestra.run.smithi040.stdout:Total 18 MB/s | 93 MB 00:05 2023-07-29T13:07:25.197 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:07:25.602 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:07:25.602 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:07:25.743 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-core-5.14.0-344.el9.x86_6 7.7 MB/s | 31 MB 00:04 2023-07-29T13:07:25.746 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:07:25.746 INFO:teuthology.orchestra.run.smithi045.stdout:Total 19 MB/s | 93 MB 00:04 2023-07-29T13:07:25.747 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:07:26.163 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:07:26.163 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:07:26.443 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:07:26.444 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:07:27.011 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:07:27.011 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:07:28.735 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:07:29.350 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:07:29.408 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-modules-core-5.14.0-344.el9.x86_64 1/4 2023-07-29T13:07:29.561 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-core-5.14.0-344.el9.x86_64 2/4 2023-07-29T13:07:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-344.el9.x86_64 1/4 2023-07-29T13:07:30.203 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-344.el9.x86_64 2/4 2023-07-29T13:07:31.865 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-core-5.14.0-344.el9.x86_64 2/4 2023-07-29T13:07:32.045 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-modules-5.14.0-344.el9.x86_64 3/4 2023-07-29T13:07:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-344.el9.x86_64 2/4 2023-07-29T13:07:32.730 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-344.el9.x86_64 3/4 2023-07-29T13:07:36.982 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-5.14.0-344.el9.x86_64 3/4 2023-07-29T13:07:37.338 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:07:37.660 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-344.el9.x86_64 3/4 2023-07-29T13:07:37.773 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:07:42.305 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-core-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:07:42.725 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:08:13.068 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-core-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:08:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:08:13.763 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:08:13.764 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-344.el9.x86_64 1/4 2023-07-29T13:08:13.764 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-core-5.14.0-344.el9.x86_64 2/4 2023-07-29T13:08:13.764 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-modules-5.14.0-344.el9.x86_64 3/4 2023-07-29T13:08:14.210 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-modules-core-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:08:14.210 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:08:14.210 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:08:14.210 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-5.14.0-344.el9.x86_64 2023-07-29T13:08:14.210 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-core-5.14.0-344.el9.x86_64 2023-07-29T13:08:14.211 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-5.14.0-344.el9.x86_64 2023-07-29T13:08:14.211 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-core-5.14.0-344.el9.x86_64 2023-07-29T13:08:14.211 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:08:14.211 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:08:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:08:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:08:14.720 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2023-07-29T13:08:14.816 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-344.el9.x86_64 2023-07-29T13:08:14.816 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-344.el9.x86_64 2023-07-29T13:08:14.816 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-239.el9.x86_64 Expected: 5.14.0-344.el9.x86_64 2023-07-29T13:08:14.817 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-07-29T13:08:14.817 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-07-29T13:08:14.817 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-07-29T13:08:14.817 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi040.front.sepia.ceph.com, path=None, version=distro) 2023-07-29T13:08:14.817 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2023-07-29T13:08:14.887 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-344.el9.x86_64 2023-07-29T13:08:14.888 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-344.el9.x86_64 2023-07-29T13:08:14.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo rpm -qi grub2-tools 2023-07-29T13:08:14.925 INFO:teuthology.orchestra.run.smithi040.stdout:Name : grub2-tools 2023-07-29T13:08:14.925 INFO:teuthology.orchestra.run.smithi040.stdout:Epoch : 1 2023-07-29T13:08:14.925 INFO:teuthology.orchestra.run.smithi040.stdout:Version : 2.06 2023-07-29T13:08:14.926 INFO:teuthology.orchestra.run.smithi040.stdout:Release : 53.el9 2023-07-29T13:08:14.926 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: x86_64 2023-07-29T13:08:14.926 INFO:teuthology.orchestra.run.smithi040.stdout:Install Date: Mon 30 Jan 2023 10:38:40 AM UTC 2023-07-29T13:08:14.926 INFO:teuthology.orchestra.run.smithi040.stdout:Group : Unspecified 2023-07-29T13:08:14.926 INFO:teuthology.orchestra.run.smithi040.stdout:Size : 8551923 2023-07-29T13:08:14.926 INFO:teuthology.orchestra.run.smithi040.stdout:License : GPLv3+ 2023-07-29T13:08:14.926 INFO:teuthology.orchestra.run.smithi040.stdout:Signature : RSA/SHA256, Wed 18 Jan 2023 04:40:07 PM UTC, Key ID 05b555b38483c65d 2023-07-29T13:08:14.927 INFO:teuthology.orchestra.run.smithi040.stdout:Source RPM : grub2-2.06-53.el9.src.rpm 2023-07-29T13:08:14.927 INFO:teuthology.orchestra.run.smithi040.stdout:Build Date : Fri 13 Jan 2023 09:07:28 PM UTC 2023-07-29T13:08:14.927 INFO:teuthology.orchestra.run.smithi040.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-07-29T13:08:14.927 INFO:teuthology.orchestra.run.smithi040.stdout:Packager : builder@centos.org 2023-07-29T13:08:14.927 INFO:teuthology.orchestra.run.smithi040.stdout:Vendor : CentOS 2023-07-29T13:08:14.927 INFO:teuthology.orchestra.run.smithi040.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-29T13:08:14.927 INFO:teuthology.orchestra.run.smithi040.stdout:Summary : Support tools for GRUB. 2023-07-29T13:08:14.927 INFO:teuthology.orchestra.run.smithi040.stdout:Description : 2023-07-29T13:08:14.928 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:08:14.928 INFO:teuthology.orchestra.run.smithi040.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-29T13:08:14.928 INFO:teuthology.orchestra.run.smithi040.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-29T13:08:14.928 INFO:teuthology.orchestra.run.smithi040.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-29T13:08:14.928 INFO:teuthology.orchestra.run.smithi040.stdout:hardware devices. 2023-07-29T13:08:14.928 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:08:14.928 INFO:teuthology.orchestra.run.smithi040.stdout:This subpackage provides tools for support of all platforms. 2023-07-29T13:08:14.929 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-29T13:08:14.929 INFO:teuthology.task.kernel:Updating grub on smithi040 to boot 5.14.0-344.el9.x86_64 2023-07-29T13:08:14.930 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-29T13:08:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:08:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-344.el9.x86_64 1/4 2023-07-29T13:08:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-344.el9.x86_64 2/4 2023-07-29T13:08:14.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-344.el9.x86_64 3/4 2023-07-29T13:08:15.392 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-344.el9.x86_64 4/4 2023-07-29T13:08:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:08:15.393 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:08:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-344.el9.x86_64 2023-07-29T13:08:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-344.el9.x86_64 2023-07-29T13:08:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-344.el9.x86_64 2023-07-29T13:08:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-344.el9.x86_64 2023-07-29T13:08:15.394 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:08:15.394 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:08:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2023-07-29T13:08:16.056 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2023-07-29T13:08:16.155 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-344.el9.x86_64 2023-07-29T13:08:16.156 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-344.el9.x86_64 2023-07-29T13:08:16.156 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-239.el9.x86_64 Expected: 5.14.0-344.el9.x86_64 2023-07-29T13:08:16.156 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-07-29T13:08:16.156 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-07-29T13:08:16.156 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-07-29T13:08:16.157 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2023-07-29T13:08:16.157 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2023-07-29T13:08:16.227 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-344.el9.x86_64 2023-07-29T13:08:16.227 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-344.el9.x86_64 2023-07-29T13:08:16.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2023-07-29T13:08:16.265 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2023-07-29T13:08:16.265 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2023-07-29T13:08:16.265 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2023-07-29T13:08:16.265 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 53.el9 2023-07-29T13:08:16.266 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2023-07-29T13:08:16.266 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Mon 30 Jan 2023 10:38:40 AM UTC 2023-07-29T13:08:16.266 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2023-07-29T13:08:16.266 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8551923 2023-07-29T13:08:16.266 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2023-07-29T13:08:16.266 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Wed 18 Jan 2023 04:40:07 PM UTC, Key ID 05b555b38483c65d 2023-07-29T13:08:16.266 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-53.el9.src.rpm 2023-07-29T13:08:16.267 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Fri 13 Jan 2023 09:07:28 PM UTC 2023-07-29T13:08:16.267 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-07-29T13:08:16.267 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2023-07-29T13:08:16.267 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2023-07-29T13:08:16.267 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-29T13:08:16.267 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2023-07-29T13:08:16.267 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2023-07-29T13:08:16.268 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:08:16.268 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-29T13:08:16.268 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-29T13:08:16.268 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-29T13:08:16.268 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2023-07-29T13:08:16.268 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:08:16.269 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2023-07-29T13:08:16.269 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-29T13:08:16.270 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-344.el9.x86_64 2023-07-29T13:08:16.270 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-29T13:08:17.121 INFO:teuthology.orchestra.run.smithi040.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-07-29T13:08:17.158 INFO:teuthology.orchestra.run.smithi040.stderr:done 2023-07-29T13:08:17.160 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2023-07-29T13:08:17.175 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.S9R9c5ksTm 2023-07-29T13:08:17.176 DEBUG:teuthology.orchestra.run.smithi040:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.S9R9c5ksTm 2023-07-29T13:08:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2023-07-29T13:08:17.265 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.S9R9c5ksTm 2023-07-29T13:08:17.366 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.S9R9c5ksTm is 6KB 2023-07-29T13:08:17.414 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.S9R9c5ksTm 2023-07-29T13:08:17.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo /bin/ls /boot/loader/entries || true 2023-07-29T13:08:17.497 INFO:teuthology.orchestra.run.smithi040.stdout:e13cba1d25f14c51a1d04da8db823a4e-0-rescue.conf 2023-07-29T13:08:17.497 INFO:teuthology.orchestra.run.smithi040.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-239.el9.x86_64.conf 2023-07-29T13:08:17.497 INFO:teuthology.orchestra.run.smithi040.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-344.el9.x86_64.conf 2023-07-29T13:08:17.498 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-set-default e13cba1d25f14c51a1d04da8db823a4e-5.14.0-344.el9.x86_64 2023-07-29T13:08:17.662 DEBUG:teuthology.orchestra.run.smithi040:> sudo shutdown -r now 2023-07-29T13:08:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-07-29T13:08:18.872 INFO:teuthology.orchestra.run.smithi045.stderr:done 2023-07-29T13:08:18.879 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2023-07-29T13:08:18.928 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.rTycflTaNn 2023-07-29T13:08:18.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rTycflTaNn 2023-07-29T13:08:18.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.rTycflTaNn 2023-07-29T13:08:19.091 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.rTycflTaNn is 6KB 2023-07-29T13:08:19.138 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.rTycflTaNn 2023-07-29T13:08:19.152 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2023-07-29T13:08:19.220 INFO:teuthology.orchestra.run.smithi045.stdout:e13cba1d25f14c51a1d04da8db823a4e-0-rescue.conf 2023-07-29T13:08:19.221 INFO:teuthology.orchestra.run.smithi045.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-239.el9.x86_64.conf 2023-07-29T13:08:19.221 INFO:teuthology.orchestra.run.smithi045.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-344.el9.x86_64.conf 2023-07-29T13:08:19.222 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default e13cba1d25f14c51a1d04da8db823a4e-5.14.0-344.el9.x86_64 2023-07-29T13:08:19.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2023-07-29T13:08:47.692 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-07-29T13:08:47.693 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-07-29T13:08:47.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-29T13:08:49.388 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-07-29T13:08:49.388 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-07-29T13:08:49.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-29T13:09:06.032 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.40 2023-07-29T13:09:07.824 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-07-29T13:09:15.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-07-29T13:09:15.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-29T13:09:16.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-07-29T13:09:16.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-29T13:10:15.100 DEBUG:teuthology.orchestra.remote:timed out 2023-07-29T13:10:16.832 DEBUG:teuthology.orchestra.remote:timed out 2023-07-29T13:10:27.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-07-29T13:10:27.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-29T13:10:27.360 DEBUG:teuthology.orchestra.run.smithi040:> true 2023-07-29T13:10:27.704 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-07-29T13:10:27.704 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-344.el9.x86_64"... 2023-07-29T13:10:27.704 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2023-07-29T13:10:27.719 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-344.el9.x86_64 2023-07-29T13:10:27.720 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-344.el9.x86_64 vs 5.14.0-344.el9.x86_64 2023-07-29T13:10:27.720 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-29T13:10:27.720 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-07-29T13:10:28.721 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-07-29T13:10:28.721 DEBUG:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-29T13:10:28.777 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2023-07-29T13:10:28.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-07-29T13:10:28.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-29T13:10:28.846 DEBUG:teuthology.parallel:result is None 2023-07-29T13:10:29.143 DEBUG:teuthology.orchestra.run.smithi045:> true 2023-07-29T13:10:29.500 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-07-29T13:10:29.501 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-344.el9.x86_64"... 2023-07-29T13:10:29.501 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2023-07-29T13:10:29.516 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-344.el9.x86_64 2023-07-29T13:10:29.516 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-344.el9.x86_64 vs 5.14.0-344.el9.x86_64 2023-07-29T13:10:29.517 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-29T13:10:29.517 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-07-29T13:10:30.517 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-07-29T13:10:30.518 DEBUG:teuthology.orchestra.run.smithi045:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-29T13:10:30.584 INFO:teuthology.orchestra.run.smithi045.stdout:ttyS1 2023-07-29T13:10:30.626 DEBUG:teuthology.parallel:result is None 2023-07-29T13:10:30.626 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-29T13:10:30.640 INFO:teuthology.task.internal:Creating test directory... 2023-07-29T13:10:30.641 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-29T13:10:30.644 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-29T13:10:30.661 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-29T13:10:30.668 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-29T13:10:30.674 INFO:teuthology.task.internal:Creating archive directory... 2023-07-29T13:10:30.674 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-29T13:10:30.702 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-29T13:10:30.735 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-29T13:10:30.742 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-29T13:10:30.742 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-29T13:10:30.772 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-29T13:10:30.814 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-29T13:10:30.816 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-29T13:10:30.832 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-29T13:10:30.833 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-29T13:10:30.835 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-29T13:10:30.842 INFO:teuthology.task.internal:Configuring sudo... 2023-07-29T13:10:30.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-29T13:10:30.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-29T13:10:30.910 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-29T13:10:30.919 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-29T13:10:30.920 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-29T13:10:30.952 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-29T13:10:30.968 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-29T13:10:31.055 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-29T13:10:31.132 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-29T13:10:31.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-29T13:10:31.225 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:10:31.225 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-29T13:10:31.297 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-29T13:10:31.345 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-29T13:10:31.411 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-29T13:10:31.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-29T13:10:31.503 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:10:31.503 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-29T13:10:31.575 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2023-07-29T13:10:31.578 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2023-07-29T13:10:31.635 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-29T13:10:31.664 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-29T13:10:31.968 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-29T13:10:31.977 INFO:teuthology.task.internal:Starting timer... 2023-07-29T13:10:31.977 INFO:teuthology.run_tasks:Running task pcp... 2023-07-29T13:10:31.988 INFO:teuthology.run_tasks:Running task selinux... 2023-07-29T13:10:31.998 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2023-07-29T13:10:32.072 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2023-07-29T13:10:32.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2023-07-29T13:10:32.121 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2023-07-29T13:10:32.123 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-29T13:10:32.123 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2023-07-29T13:10:32.142 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2023-07-29T13:10:32.143 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2023-07-29T13:10:32.165 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2023-07-29T13:10:32.166 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi045.front.sepia.ceph.com': 'permissive'} 2023-07-29T13:10:32.166 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-29T13:10:32.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-29T13:10:32.217 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-29T13:10:32.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-29T13:10:32.250 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-29T13:10:32.250 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2023-07-29T13:10:32.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2023-07-29T13:10:32.335 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-29T13:10:32.344 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-29T13:10:32.351 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-29T13:10:32.544 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-29T13:10:32.565 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-29T13:10:32.567 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi045.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-29T13:16:53.539 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi045.front.sepia.ceph.com')] 2023-07-29T13:16:53.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-07-29T13:16:53.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-29T13:16:53.637 DEBUG:teuthology.orchestra.run.smithi040:> true 2023-07-29T13:16:53.706 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-07-29T13:16:53.707 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-07-29T13:16:53.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-29T13:16:53.802 DEBUG:teuthology.orchestra.run.smithi045:> true 2023-07-29T13:16:53.828 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-07-29T13:16:53.829 INFO:teuthology.run_tasks:Running task clock... 2023-07-29T13:16:53.839 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-29T13:16:53.840 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-29T13:16:53.840 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-29T13:16:53.842 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-29T13:16:53.843 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-29T13:16:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-29T13:16:53.897 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-29T13:16:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-29T13:16:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-29T13:16:53.933 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2023-07-29T13:16:53.950 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2023-07-29T13:16:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2023-07-29T13:16:53.970 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-29T13:16:53.978 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2023-07-29T13:16:53.990 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-29T13:16:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-29T13:16:54.017 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-29T13:16:54.038 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2023-07-29T13:16:54.041 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-29T13:16:54.041 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2023-07-29T13:16:54.041 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-29T13:16:54.042 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-29T13:16:54.042 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-29T13:16:54.042 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-29T13:16:54.065 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2023-07-29T13:16:54.069 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-29T13:16:54.069 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2023-07-29T13:16:54.069 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-29T13:16:54.069 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-29T13:16:54.069 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-29T13:16:54.070 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-29T13:16:54.070 INFO:teuthology.run_tasks:Running task install... 2023-07-29T13:16:54.087 DEBUG:teuthology.task.install:project ceph 2023-07-29T13:16:54.087 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '0484097b0572e96f42cf6402cbe0ac8dcb046577'}} 2023-07-29T13:16:54.088 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': '0484097b0572e96f42cf6402cbe0ac8dcb046577'} 2023-07-29T13:16:54.088 INFO:teuthology.task.install:Using flavor: default 2023-07-29T13:16:54.093 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2023-07-29T13:16:54.094 INFO:teuthology.task.install:extra packages: [] 2023-07-29T13:16:54.094 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0484097b0572e96f42cf6402cbe0ac8dcb046577', 'tag': None, 'wait_for_package': False} 2023-07-29T13:16:54.095 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0484097b0572e96f42cf6402cbe0ac8dcb046577 2023-07-29T13:16:54.097 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0484097b0572e96f42cf6402cbe0ac8dcb046577', 'tag': None, 'wait_for_package': False} 2023-07-29T13:16:54.097 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0484097b0572e96f42cf6402cbe0ac8dcb046577 2023-07-29T13:16:54.384 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/reef-release/0484097b0572e96f42cf6402cbe0ac8dcb046577/centos/9/flavors/default/ 2023-07-29T13:16:54.384 INFO:teuthology.task.install.rpm:Package version is 18.1.3-11.g0484097b 2023-07-29T13:16:54.391 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/reef-release/0484097b0572e96f42cf6402cbe0ac8dcb046577/centos/9/flavors/default/ 2023-07-29T13:16:54.391 INFO:teuthology.task.install.rpm:Package version is 18.1.3-11.g0484097b 2023-07-29T13:16:54.605 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/reef-release/0484097b0572e96f42cf6402cbe0ac8dcb046577/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/reef-release/0484097b0572e96f42cf6402cbe0ac8dcb046577/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/reef-release/0484097b0572e96f42cf6402cbe0ac8dcb046577/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-29T13:16:54.605 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:16:54.605 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-29T13:16:54.612 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/reef-release/0484097b0572e96f42cf6402cbe0ac8dcb046577/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/reef-release/0484097b0572e96f42cf6402cbe0ac8dcb046577/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/reef-release/0484097b0572e96f42cf6402cbe0ac8dcb046577/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-29T13:16:54.612 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:16:54.612 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-29T13:16:54.639 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2023-07-29T13:16:54.639 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0484097b0572e96f42cf6402cbe0ac8dcb046577/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-29T13:16:54.645 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2023-07-29T13:16:54.645 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0484097b0572e96f42cf6402cbe0ac8dcb046577/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-29T13:16:54.711 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-29T13:16:54.718 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-29T13:16:54.800 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-29T13:16:54.805 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-29T13:16:54.830 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2023-07-29T13:16:54.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2023-07-29T13:16:54.835 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2023-07-29T13:16:54.836 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2023-07-29T13:16:55.076 INFO:teuthology.orchestra.run.smithi045.stdout:71 files removed 2023-07-29T13:16:55.081 INFO:teuthology.orchestra.run.smithi040.stdout:71 files removed 2023-07-29T13:16:55.103 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw 2023-07-29T13:16:55.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw 2023-07-29T13:16:55.991 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 132 kB/s | 77 kB 00:00 2023-07-29T13:16:56.000 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 130 kB/s | 77 kB 00:00 2023-07-29T13:16:56.434 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 23 kB/s | 9.4 kB 00:00 2023-07-29T13:16:56.451 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 23 kB/s | 9.4 kB 00:00 2023-07-29T13:16:56.877 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2023-07-29T13:16:56.884 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2023-07-29T13:16:57.321 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - AppStream 15 MB/s | 6.3 MB 00:00 2023-07-29T13:16:57.370 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-07-29T13:16:58.730 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2023-07-29T13:16:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2023-07-29T13:17:02.520 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 7.2 MB 00:03 2023-07-29T13:17:02.886 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 7.2 MB 00:03 2023-07-29T13:17:05.204 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 14 MB/s | 17 MB 00:01 2023-07-29T13:17:05.641 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 17 MB 00:02 2023-07-29T13:17:09.497 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 5.6 MB 00:00 2023-07-29T13:17:09.930 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 5.6 MB 00:00 2023-07-29T13:17:11.591 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 13 kB 00:01 2023-07-29T13:17:12.806 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 7.5 kB/s | 13 kB 00:01 2023-07-29T13:17:14.485 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 12 MB/s | 18 MB 00:01 2023-07-29T13:17:20.741 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 2.0 MB/s | 18 MB 00:09 2023-07-29T13:17:21.276 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:17:21.294 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:17:21.294 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-29T13:17:21.295 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:17:21.295 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:17:21.295 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:18.1.3-11.g0484097b.el9 ceph 7.6 M 2023-07-29T13:17:21.295 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2023-07-29T13:17:21.295 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:18.1.3-11.g0484097b.el9 ceph 3.3 M 2023-07-29T13:17:21.295 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:18.1.3-11.g0484097b.el9 ceph 3.0 M 2023-07-29T13:17:21.295 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-29T13:17:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-07-29T13:17:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:18.1.3-11.g0484097b.el9 ceph 5.2 M 2023-07-29T13:17:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:18.1.3-11.g0484097b.el9 ceph 18 M 2023-07-29T13:17:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:18.1.3-11.g0484097b.el9 ceph 24 k 2023-07-29T13:17:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-07-29T13:17:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-07-29T13:17:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-07-29T13:17:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-07-29T13:17:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:18.1.3-11.g0484097b.el9 ceph 653 k 2023-07-29T13:17:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-07-29T13:17:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-07-29T13:17:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-07-29T13:17:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:18.1.3-11.g0484097b.el9 ceph 474 k 2023-07-29T13:17:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-07-29T13:17:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:18.1.3-11.g0484097b.el9 ceph 4.4 M 2023-07-29T13:17:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-07-29T13:17:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-07-29T13:17:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-07-29T13:17:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:18.1.3-11.g0484097b.el9 ceph 45 k 2023-07-29T13:17:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:18.1.3-11.g0484097b.el9 ceph 120 k 2023-07-29T13:17:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:18.1.3-11.g0484097b.el9 ceph 156 k 2023-07-29T13:17:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-07-29T13:17:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:18.1.3-11.g0484097b.el9 ceph 320 k 2023-07-29T13:17:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:18.1.3-11.g0484097b.el9 ceph 297 k 2023-07-29T13:17:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:18.1.3-11.g0484097b.el9 ceph 99 k 2023-07-29T13:17:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: re2 x86_64 1:20211101-2.el9 epel 192 k 2023-07-29T13:17:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-07-29T13:17:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:17:21.299 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:17:21.299 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:17:21.300 INFO:teuthology.orchestra.run.smithi040.stdout:Install 28 Packages 2023-07-29T13:17:21.300 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 2 Packages 2023-07-29T13:17:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:17:21.300 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 53 M 2023-07-29T13:17:21.301 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:17:23.836 INFO:teuthology.orchestra.run.smithi040.stdout:(1/30): ceph-base-18.1.3-11.g0484097b.el9.x86_6 2.2 MB/s | 5.2 MB 00:02 2023-07-29T13:17:23.902 INFO:teuthology.orchestra.run.smithi040.stdout:(2/30): ceph-selinux-18.1.3-11.g0484097b.el9.x8 363 kB/s | 24 kB 00:00 2023-07-29T13:17:24.136 INFO:teuthology.orchestra.run.smithi040.stdout:(3/30): libcephfs2-18.1.3-11.g0484097b.el9.x86_ 2.7 MB/s | 653 kB 00:00 2023-07-29T13:17:24.320 INFO:teuthology.orchestra.run.smithi040.stdout:(4/30): libradosstriper1-18.1.3-11.g0484097b.el 2.5 MB/s | 474 kB 00:00 2023-07-29T13:17:24.437 INFO:teuthology.orchestra.run.smithi040.stdout:(5/30): ceph-radosgw-18.1.3-11.g0484097b.el9.x8 2.6 MB/s | 7.6 MB 00:02 2023-07-29T13:17:24.521 INFO:teuthology.orchestra.run.smithi040.stdout:(6/30): python3-ceph-argparse-18.1.3-11.g048409 541 kB/s | 45 kB 00:00 2023-07-29T13:17:24.613 INFO:teuthology.orchestra.run.smithi040.stdout:(7/30): python3-ceph-common-18.1.3-11.g0484097b 1.3 MB/s | 120 kB 00:00 2023-07-29T13:17:24.704 INFO:teuthology.orchestra.run.smithi040.stdout:(8/30): python3-cephfs-18.1.3-11.g0484097b.el9. 1.7 MB/s | 156 kB 00:00 2023-07-29T13:17:24.813 INFO:teuthology.orchestra.run.smithi040.stdout:(9/30): python3-rados-18.1.3-11.g0484097b.el9.x 2.9 MB/s | 320 kB 00:00 2023-07-29T13:17:24.930 INFO:teuthology.orchestra.run.smithi040.stdout:(10/30): python3-rbd-18.1.3-11.g0484097b.el9.x8 2.5 MB/s | 297 kB 00:00 2023-07-29T13:17:25.047 INFO:teuthology.orchestra.run.smithi040.stdout:(11/30): librgw2-18.1.3-11.g0484097b.el9.x86_64 6.1 MB/s | 4.4 MB 00:00 2023-07-29T13:17:25.089 INFO:teuthology.orchestra.run.smithi040.stdout:(12/30): python3-rgw-18.1.3-11.g0484097b.el9.x8 627 kB/s | 99 kB 00:00 2023-07-29T13:17:25.214 INFO:teuthology.orchestra.run.smithi040.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 634 kB/s | 106 kB 00:00 2023-07-29T13:17:25.248 INFO:teuthology.orchestra.run.smithi040.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2023-07-29T13:17:25.273 INFO:teuthology.orchestra.run.smithi040.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 759 kB/s | 44 kB 00:00 2023-07-29T13:17:25.490 INFO:teuthology.orchestra.run.smithi040.stdout:(16/30): ceph-common-18.1.3-11.g0484097b.el9.x8 4.5 MB/s | 18 MB 00:04 2023-07-29T13:17:25.524 INFO:teuthology.orchestra.run.smithi040.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 661 kB 00:00 2023-07-29T13:17:25.566 INFO:teuthology.orchestra.run.smithi040.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 968 kB/s | 282 kB 00:00 2023-07-29T13:17:25.591 INFO:teuthology.orchestra.run.smithi040.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 416 kB/s | 41 kB 00:00 2023-07-29T13:17:26.116 INFO:teuthology.orchestra.run.smithi040.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 201 kB/s | 111 kB 00:00 2023-07-29T13:17:26.233 INFO:teuthology.orchestra.run.smithi040.stdout:(21/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 209 kB/s | 24 kB 00:00 2023-07-29T13:17:26.267 INFO:teuthology.orchestra.run.smithi040.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 441 kB/s | 309 kB 00:00 2023-07-29T13:17:26.359 INFO:teuthology.orchestra.run.smithi040.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 389 kB/s | 49 kB 00:00 2023-07-29T13:17:26.414 INFO:teuthology.orchestra.run.smithi040.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 458 kB/s | 67 kB 00:00 2023-07-29T13:17:26.667 INFO:teuthology.orchestra.run.smithi040.stdout:(25/30): libarrow-9.0.0-9.el9.x86_64.rpm 4.1 MB/s | 4.4 MB 00:01 2023-07-29T13:17:26.709 INFO:teuthology.orchestra.run.smithi040.stdout:(26/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.3 MB/s | 837 kB 00:00 2023-07-29T13:17:26.743 INFO:teuthology.orchestra.run.smithi040.stdout:(27/30): re2-20211101-2.el9.x86_64.rpm 587 kB/s | 192 kB 00:00 2023-07-29T13:17:26.910 INFO:teuthology.orchestra.run.smithi040.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2023-07-29T13:17:27.194 INFO:teuthology.orchestra.run.smithi040.stdout:(29/30): librbd1-18.1.3-11.g0484097b.el9.x86_64 6.7 MB/s | 3.0 MB 00:00 2023-07-29T13:17:27.319 INFO:teuthology.orchestra.run.smithi040.stdout:(30/30): librados2-18.1.3-11.g0484097b.el9.x86_ 5.4 MB/s | 3.3 MB 00:00 2023-07-29T13:17:27.326 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:17:27.326 INFO:teuthology.orchestra.run.smithi040.stdout:Total 8.7 MB/s | 53 MB 00:06 2023-07-29T13:17:27.327 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:17:27.368 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:17:27.369 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:17:27.452 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:17:27.463 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:17:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-07-29T13:17:27.464 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:17:27.464 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:17:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:18.1.3-11.g0484097b.el9 ceph 7.6 M 2023-07-29T13:17:27.464 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2023-07-29T13:17:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:18.1.3-11.g0484097b.el9 ceph 3.3 M 2023-07-29T13:17:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:18.1.3-11.g0484097b.el9 ceph 3.0 M 2023-07-29T13:17:27.465 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-07-29T13:17:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-07-29T13:17:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:18.1.3-11.g0484097b.el9 ceph 5.2 M 2023-07-29T13:17:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:18.1.3-11.g0484097b.el9 ceph 18 M 2023-07-29T13:17:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:18.1.3-11.g0484097b.el9 ceph 24 k 2023-07-29T13:17:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-07-29T13:17:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-07-29T13:17:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-07-29T13:17:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-07-29T13:17:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:18.1.3-11.g0484097b.el9 ceph 653 k 2023-07-29T13:17:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-07-29T13:17:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-07-29T13:17:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-07-29T13:17:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:18.1.3-11.g0484097b.el9 ceph 474 k 2023-07-29T13:17:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-07-29T13:17:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:18.1.3-11.g0484097b.el9 ceph 4.4 M 2023-07-29T13:17:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-07-29T13:17:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-07-29T13:17:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-07-29T13:17:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:18.1.3-11.g0484097b.el9 ceph 45 k 2023-07-29T13:17:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:18.1.3-11.g0484097b.el9 ceph 120 k 2023-07-29T13:17:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:18.1.3-11.g0484097b.el9 ceph 156 k 2023-07-29T13:17:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-07-29T13:17:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:18.1.3-11.g0484097b.el9 ceph 320 k 2023-07-29T13:17:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:18.1.3-11.g0484097b.el9 ceph 297 k 2023-07-29T13:17:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:18.1.3-11.g0484097b.el9 ceph 99 k 2023-07-29T13:17:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-2.el9 epel 192 k 2023-07-29T13:17:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-07-29T13:17:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:17:27.471 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:17:27.471 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:17:27.471 INFO:teuthology.orchestra.run.smithi045.stdout:Install 28 Packages 2023-07-29T13:17:27.471 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 2 Packages 2023-07-29T13:17:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:17:27.472 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 53 M 2023-07-29T13:17:27.473 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:17:27.843 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:17:27.843 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:17:28.433 INFO:teuthology.orchestra.run.smithi045.stdout:(1/30): ceph-base-18.1.3-11.g0484097b.el9.x86_6 5.9 MB/s | 5.2 MB 00:00 2023-07-29T13:17:28.517 INFO:teuthology.orchestra.run.smithi045.stdout:(2/30): ceph-selinux-18.1.3-11.g0484097b.el9.x8 289 kB/s | 24 kB 00:00 2023-07-29T13:17:28.642 INFO:teuthology.orchestra.run.smithi045.stdout:(3/30): libcephfs2-18.1.3-11.g0484097b.el9.x86_ 5.1 MB/s | 653 kB 00:00 2023-07-29T13:17:28.767 INFO:teuthology.orchestra.run.smithi045.stdout:(4/30): libradosstriper1-18.1.3-11.g0484097b.el 3.7 MB/s | 474 kB 00:00 2023-07-29T13:17:29.009 INFO:teuthology.orchestra.run.smithi045.stdout:(5/30): ceph-radosgw-18.1.3-11.g0484097b.el9.x8 5.3 MB/s | 7.6 MB 00:01 2023-07-29T13:17:29.114 INFO:teuthology.orchestra.run.smithi045.stdout:(6/30): python3-ceph-argparse-18.1.3-11.g048409 432 kB/s | 45 kB 00:00 2023-07-29T13:17:29.163 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:17:29.261 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-07-29T13:17:29.281 INFO:teuthology.orchestra.run.smithi045.stdout:(7/30): librgw2-18.1.3-11.g0484097b.el9.x86_64. 8.6 MB/s | 4.4 MB 00:00 2023-07-29T13:17:29.307 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2023-07-29T13:17:29.314 INFO:teuthology.orchestra.run.smithi045.stdout:(8/30): python3-ceph-common-18.1.3-11.g0484097b 597 kB/s | 120 kB 00:00 2023-07-29T13:17:29.366 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2023-07-29T13:17:29.540 INFO:teuthology.orchestra.run.smithi045.stdout:(9/30): ceph-common-18.1.3-11.g0484097b.el9.x86 9.2 MB/s | 18 MB 00:01 2023-07-29T13:17:29.573 INFO:teuthology.orchestra.run.smithi045.stdout:(10/30): python3-cephfs-18.1.3-11.g0484097b.el9 534 kB/s | 156 kB 00:00 2023-07-29T13:17:29.615 INFO:teuthology.orchestra.run.smithi045.stdout:(11/30): python3-rados-18.1.3-11.g0484097b.el9. 1.0 MB/s | 320 kB 00:00 2023-07-29T13:17:29.657 INFO:teuthology.orchestra.run.smithi045.stdout:(12/30): python3-rgw-18.1.3-11.g0484097b.el9.x8 1.2 MB/s | 99 kB 00:00 2023-07-29T13:17:29.691 INFO:teuthology.orchestra.run.smithi045.stdout:(13/30): python3-rbd-18.1.3-11.g0484097b.el9.x8 1.9 MB/s | 297 kB 00:00 2023-07-29T13:17:29.778 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2023-07-29T13:17:29.814 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:18.1.3-11.g0484097b.el9.x86_64 5/32 2023-07-29T13:17:29.833 INFO:teuthology.orchestra.run.smithi045.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 487 kB/s | 106 kB 00:00 2023-07-29T13:17:29.866 INFO:teuthology.orchestra.run.smithi045.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 762 kB/s | 159 kB 00:00 2023-07-29T13:17:29.892 INFO:teuthology.orchestra.run.smithi045.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 294 kB/s | 44 kB 00:00 2023-07-29T13:17:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:18.1.3-11.g0484097b.el9.x86_64 5/32 2023-07-29T13:17:29.950 INFO:teuthology.orchestra.run.smithi045.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 661 kB 00:00 2023-07-29T13:17:29.984 INFO:teuthology.orchestra.run.smithi045.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 282 kB 00:00 2023-07-29T13:17:29.997 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:18.1.3-11.g0484097b.el9.x86_64 6/32 2023-07-29T13:17:30.009 INFO:teuthology.orchestra.run.smithi045.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 352 kB/s | 41 kB 00:00 2023-07-29T13:17:30.033 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:18.1.3-11.g0484097b.el9.x86_64 7/32 2023-07-29T13:17:30.096 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:18.1.3-11.g0484097b.el9.x86_64 7/32 2023-07-29T13:17:30.133 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2023-07-29T13:17:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:18.1.3-11.g0484097b.el9.x8 9/32 2023-07-29T13:17:30.260 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:18.1.3-11.g0484097b.el9.x86_64 10/32 2023-07-29T13:17:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:18.1.3-11.g0484097b.el9.x86_64 11/32 2023-07-29T13:17:30.351 INFO:teuthology.orchestra.run.smithi045.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 277 kB/s | 111 kB 00:00 2023-07-29T13:17:30.368 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:18.1.3-11.g0484097b.el9.x86_64 11/32 2023-07-29T13:17:30.393 INFO:teuthology.orchestra.run.smithi045.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 805 kB/s | 309 kB 00:00 2023-07-29T13:17:30.493 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : re2-1:20211101-2.el9.x86_64 12/32 2023-07-29T13:17:30.536 INFO:teuthology.orchestra.run.smithi045.stdout:(22/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 132 kB/s | 24 kB 00:00 2023-07-29T13:17:30.543 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-07-29T13:17:30.552 INFO:teuthology.orchestra.run.smithi045.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 305 kB/s | 49 kB 00:00 2023-07-29T13:17:30.591 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-07-29T13:17:30.632 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-07-29T13:17:30.653 INFO:teuthology.orchestra.run.smithi045.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2023-07-29T13:17:30.668 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-07-29T13:17:30.712 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-07-29T13:17:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-07-29T13:17:30.764 INFO:teuthology.orchestra.run.smithi045.stdout:(25/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.9 MB/s | 837 kB 00:00 2023-07-29T13:17:30.847 INFO:teuthology.orchestra.run.smithi045.stdout:(26/30): libarrow-9.0.0-9.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2023-07-29T13:17:30.881 INFO:teuthology.orchestra.run.smithi045.stdout:(27/30): re2-20211101-2.el9.x86_64.rpm 845 kB/s | 192 kB 00:00 2023-07-29T13:17:30.956 INFO:teuthology.orchestra.run.smithi045.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2023-07-29T13:17:31.075 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-07-29T13:17:31.116 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:18.1.3-11.g0484097b.el9.x86_64 20/32 2023-07-29T13:17:31.190 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:18.1.3-11.g0484097b.el9.x86_64 20/32 2023-07-29T13:17:31.237 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:18.1.3-11.g0484097b.el9.x86_64 21/32 2023-07-29T13:17:31.290 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-07-29T13:17:31.306 INFO:teuthology.orchestra.run.smithi045.stdout:(29/30): librbd1-18.1.3-11.g0484097b.el9.x86_64 7.1 MB/s | 3.0 MB 00:00 2023-07-29T13:17:31.381 INFO:teuthology.orchestra.run.smithi045.stdout:(30/30): librados2-18.1.3-11.g0484097b.el9.x86_ 6.1 MB/s | 3.3 MB 00:00 2023-07-29T13:17:31.386 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:17:31.386 INFO:teuthology.orchestra.run.smithi045.stdout:Total 13 MB/s | 53 MB 00:03 2023-07-29T13:17:31.387 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:17:31.429 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:17:31.430 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:17:31.713 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-07-29T13:17:31.752 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:18.1.3-11.g0484097b.el9.x86_64 24/32 2023-07-29T13:17:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:18.1.3-11.g0484097b.el9.x86_64 24/32 2023-07-29T13:17:31.903 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:17:31.904 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:17:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:18.1.3-11.g0484097b.el9.x86_64 25/32 2023-07-29T13:17:31.971 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:18.1.3-11.g0484097b.el9.x86_ 26/32 2023-07-29T13:17:33.265 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:17:33.361 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-07-29T13:17:33.411 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2023-07-29T13:17:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2023-07-29T13:17:33.890 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2023-07-29T13:17:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:18.1.3-11.g0484097b.el9.x86_64 5/32 2023-07-29T13:17:34.061 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:18.1.3-11.g0484097b.el9.x86_64 5/32 2023-07-29T13:17:34.160 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:18.1.3-11.g0484097b.el9.x86_64 6/32 2023-07-29T13:17:34.204 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:18.1.3-11.g0484097b.el9.x86_64 7/32 2023-07-29T13:17:34.283 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:18.1.3-11.g0484097b.el9.x86_64 7/32 2023-07-29T13:17:34.302 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 27/32 2023-07-29T13:17:34.329 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2023-07-29T13:17:34.342 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 27/32 2023-07-29T13:17:34.381 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:18.1.3-11.g0484097b.el9.x8 9/32 2023-07-29T13:17:34.457 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:18.1.3-11.g0484097b.el9.x86_64 10/32 2023-07-29T13:17:34.498 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:18.1.3-11.g0484097b.el9.x86_64 11/32 2023-07-29T13:17:34.564 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:18.1.3-11.g0484097b.el9.x86_64 11/32 2023-07-29T13:17:34.692 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-2.el9.x86_64 12/32 2023-07-29T13:17:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-07-29T13:17:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-07-29T13:17:34.837 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-07-29T13:17:34.881 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-07-29T13:17:34.919 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 27/32 2023-07-29T13:17:34.933 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-07-29T13:17:34.971 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:18.1.3-11.g0484097b.el9.x86_64 28/32 2023-07-29T13:17:34.982 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-07-29T13:17:35.057 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:18.1.3-11.g0484097b.el9.x86_64 28/32 2023-07-29T13:17:35.057 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-07-29T13:17:35.057 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-07-29T13:17:35.058 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:17:35.089 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:18.1.3-11.g0484097b.el9.x86_64 29/32 2023-07-29T13:17:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-07-29T13:17:35.354 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:18.1.3-11.g0484097b.el9.x86_64 20/32 2023-07-29T13:17:35.444 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:18.1.3-11.g0484097b.el9.x86_64 20/32 2023-07-29T13:17:35.483 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:18.1.3-11.g0484097b.el9.x86_64 21/32 2023-07-29T13:17:35.529 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-07-29T13:17:35.954 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-07-29T13:17:35.998 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:18.1.3-11.g0484097b.el9.x86_64 24/32 2023-07-29T13:17:36.086 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:18.1.3-11.g0484097b.el9.x86_64 24/32 2023-07-29T13:17:36.181 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:18.1.3-11.g0484097b.el9.x86_64 25/32 2023-07-29T13:17:36.225 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:18.1.3-11.g0484097b.el9.x86_ 26/32 2023-07-29T13:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 27/32 2023-07-29T13:17:38.522 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 27/32 2023-07-29T13:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 27/32 2023-07-29T13:17:39.134 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:18.1.3-11.g0484097b.el9.x86_64 28/32 2023-07-29T13:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:18.1.3-11.g0484097b.el9.x86_64 28/32 2023-07-29T13:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-07-29T13:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-07-29T13:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:17:39.293 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:18.1.3-11.g0484097b.el9.x86_64 29/32 2023-07-29T13:17:45.231 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:18.1.3-11.g0484097b.el9.x86_64 29/32 2023-07-29T13:17:45.231 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2023-07-29T13:17:45.232 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2023-07-29T13:17:45.232 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2023-07-29T13:17:45.232 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2023-07-29T13:17:45.232 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2023-07-29T13:17:45.232 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2023-07-29T13:17:45.232 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2023-07-29T13:17:45.232 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:17:45.996 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:18.1.3-11.g0484097b.el9.x86_64 30/32 2023-07-29T13:17:46.021 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:18.1.3-11.g0484097b.el9.x86_64 30/32 2023-07-29T13:17:46.021 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:17:46.022 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-07-29T13:17:46.022 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-07-29T13:17:46.022 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-07-29T13:17:46.022 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:17:46.023 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-07-29T13:17:46.115 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-07-29T13:17:46.115 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-07-29T13:17:47.482 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-07-29T13:17:47.483 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:18.1.3-11.g0484097b.el9.x86_64 1/32 2023-07-29T13:17:47.483 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 2/32 2023-07-29T13:17:47.483 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:18.1.3-11.g0484097b.el9.x86_64 3/32 2023-07-29T13:17:47.483 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:18.1.3-11.g0484097b.el9.x86_64 4/32 2023-07-29T13:17:47.483 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:18.1.3-11.g0484097b.el9.x86_64 5/32 2023-07-29T13:17:47.483 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:18.1.3-11.g0484097b.el9.x86_64 6/32 2023-07-29T13:17:47.483 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:18.1.3-11.g0484097b.el9.x86_64 7/32 2023-07-29T13:17:47.484 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:18.1.3-11.g0484097b.el9.x8 8/32 2023-07-29T13:17:47.484 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:18.1.3-11.g0484097b.el9.x86_ 9/32 2023-07-29T13:17:47.484 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:18.1.3-11.g0484097b.el9.x86_64 10/32 2023-07-29T13:17:47.484 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:18.1.3-11.g0484097b.el9.x86_64 11/32 2023-07-29T13:17:47.484 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:18.1.3-11.g0484097b.el9.x86_64 12/32 2023-07-29T13:17:47.484 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:18.1.3-11.g0484097b.el9.x86_64 13/32 2023-07-29T13:17:47.484 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-07-29T13:17:47.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-07-29T13:17:47.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2023-07-29T13:17:47.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2023-07-29T13:17:47.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2023-07-29T13:17:47.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2023-07-29T13:17:47.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-07-29T13:17:47.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-07-29T13:17:47.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-07-29T13:17:47.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-07-29T13:17:47.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-07-29T13:17:47.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-07-29T13:17:47.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-07-29T13:17:47.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : re2-1:20211101-2.el9.x86_64 27/32 2023-07-29T13:17:47.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-07-29T13:17:47.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:18.1.3-11.g0484097b.el9.x86_64 29/32 2023-07-29T13:17:47.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-07-29T13:17:47.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:18.1.3-11.g0484097b.el9.x86_64 31/32 2023-07-29T13:17:48.207 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-07-29T13:17:48.207 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:17:48.207 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2023-07-29T13:17:48.208 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.208 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.208 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:17:48.208 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-07-29T13:17:48.208 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.208 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.208 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: fmt-8.1.1-5.el9.x86_64 2023-07-29T13:17:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-07-29T13:17:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-9.0.0-9.el9.x86_64 2023-07-29T13:17:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-07-29T13:17:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2023-07-29T13:17:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-07-29T13:17:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-07-29T13:17:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-07-29T13:17:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2023-07-29T13:17:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-07-29T13:17:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-07-29T13:17:48.211 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.211 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.211 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.211 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-07-29T13:17:48.211 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.211 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.211 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:48.211 INFO:teuthology.orchestra.run.smithi040.stdout: re2-1:20211101-2.el9.x86_64 2023-07-29T13:17:48.212 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2023-07-29T13:17:48.212 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:17:48.212 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:17:48.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-test 2023-07-29T13:17:49.148 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:36 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:17:49.284 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:17:49.285 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:17:49.285 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-29T13:17:49.286 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:17:49.286 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:17:49.286 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:18.1.3-11.g0484097b.el9 ceph 40 M 2023-07-29T13:17:49.286 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-29T13:17:49.286 INFO:teuthology.orchestra.run.smithi040.stdout: jq x86_64 1.6-15.el9 CentOS-AppStream 186 k 2023-07-29T13:17:49.286 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:18.1.3-11.g0484097b.el9 ceph 161 k 2023-07-29T13:17:49.286 INFO:teuthology.orchestra.run.smithi040.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-07-29T13:17:49.287 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-07-29T13:17:49.287 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-07-29T13:17:49.287 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:17:49.287 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:17:49.287 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:17:49.287 INFO:teuthology.orchestra.run.smithi040.stdout:Install 6 Packages 2023-07-29T13:17:49.287 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:17:49.288 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 40 M 2023-07-29T13:17:49.288 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 167 M 2023-07-29T13:17:49.288 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:17:49.461 INFO:teuthology.orchestra.run.smithi040.stdout:(1/6): jq-1.6-15.el9.x86_64.rpm 1.0 MB/s | 186 kB 00:00 2023-07-29T13:17:49.507 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:18.1.3-11.g0484097b.el9.x86_64 29/32 2023-07-29T13:17:49.507 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2023-07-29T13:17:49.507 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2023-07-29T13:17:49.508 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2023-07-29T13:17:49.508 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2023-07-29T13:17:49.508 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2023-07-29T13:17:49.508 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2023-07-29T13:17:49.508 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2023-07-29T13:17:49.508 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:17:49.520 INFO:teuthology.orchestra.run.smithi040.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2023-07-29T13:17:49.570 INFO:teuthology.orchestra.run.smithi040.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2023-07-29T13:17:49.612 INFO:teuthology.orchestra.run.smithi040.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2023-07-29T13:17:49.729 INFO:teuthology.orchestra.run.smithi040.stdout:(5/6): libcephsqlite-18.1.3-11.g0484097b.el9.x8 365 kB/s | 161 kB 00:00 2023-07-29T13:17:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:18.1.3-11.g0484097b.el9.x86_64 30/32 2023-07-29T13:17:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:18.1.3-11.g0484097b.el9.x86_64 30/32 2023-07-29T13:17:50.318 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:17:50.318 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-07-29T13:17:50.318 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-07-29T13:17:50.318 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-07-29T13:17:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:17:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-07-29T13:17:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-07-29T13:17:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-07-29T13:17:51.529 INFO:teuthology.orchestra.run.smithi040.stdout:(6/6): ceph-test-18.1.3-11.g0484097b.el9.x86_64 18 MB/s | 40 MB 00:02 2023-07-29T13:17:51.532 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:17:51.533 INFO:teuthology.orchestra.run.smithi040.stdout:Total 18 MB/s | 40 MB 00:02 2023-07-29T13:17:51.533 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:17:51.556 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:17:51.556 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:17:51.759 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-07-29T13:17:51.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:18.1.3-11.g0484097b.el9.x86_64 1/32 2023-07-29T13:17:51.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 2/32 2023-07-29T13:17:51.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:18.1.3-11.g0484097b.el9.x86_64 3/32 2023-07-29T13:17:51.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:18.1.3-11.g0484097b.el9.x86_64 4/32 2023-07-29T13:17:51.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:18.1.3-11.g0484097b.el9.x86_64 5/32 2023-07-29T13:17:51.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:18.1.3-11.g0484097b.el9.x86_64 6/32 2023-07-29T13:17:51.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:18.1.3-11.g0484097b.el9.x86_64 7/32 2023-07-29T13:17:51.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:18.1.3-11.g0484097b.el9.x8 8/32 2023-07-29T13:17:51.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:18.1.3-11.g0484097b.el9.x86_ 9/32 2023-07-29T13:17:51.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:18.1.3-11.g0484097b.el9.x86_64 10/32 2023-07-29T13:17:51.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:18.1.3-11.g0484097b.el9.x86_64 11/32 2023-07-29T13:17:51.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:18.1.3-11.g0484097b.el9.x86_64 12/32 2023-07-29T13:17:51.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:18.1.3-11.g0484097b.el9.x86_64 13/32 2023-07-29T13:17:51.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-07-29T13:17:51.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-07-29T13:17:51.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2023-07-29T13:17:51.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2023-07-29T13:17:51.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2023-07-29T13:17:51.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2023-07-29T13:17:51.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-07-29T13:17:51.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-07-29T13:17:51.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-07-29T13:17:51.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-07-29T13:17:51.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-07-29T13:17:51.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-07-29T13:17:51.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-07-29T13:17:51.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-2.el9.x86_64 27/32 2023-07-29T13:17:51.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-07-29T13:17:51.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:18.1.3-11.g0484097b.el9.x86_64 29/32 2023-07-29T13:17:51.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-07-29T13:17:51.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:18.1.3-11.g0484097b.el9.x86_64 31/32 2023-07-29T13:17:51.918 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:17:51.918 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:17:52.475 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:17:52.589 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2023-07-29T13:17:52.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-07-29T13:17:52.603 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:17:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2023-07-29T13:17:52.603 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.603 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.604 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:17:52.604 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-07-29T13:17:52.604 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.604 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.604 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.604 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.605 INFO:teuthology.orchestra.run.smithi045.stdout: fmt-8.1.1-5.el9.x86_64 2023-07-29T13:17:52.605 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-07-29T13:17:52.605 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-9.el9.x86_64 2023-07-29T13:17:52.605 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-07-29T13:17:52.605 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.605 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2023-07-29T13:17:52.605 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-07-29T13:17:52.605 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-07-29T13:17:52.606 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.606 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-07-29T13:17:52.606 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.606 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2023-07-29T13:17:52.606 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-07-29T13:17:52.606 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-07-29T13:17:52.606 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.607 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.607 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.607 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-07-29T13:17:52.607 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.607 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.607 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:17:52.607 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-2.el9.x86_64 2023-07-29T13:17:52.607 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2023-07-29T13:17:52.608 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:17:52.608 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:17:52.647 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2023-07-29T13:17:52.687 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-07-29T13:17:52.738 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : jq-1.6-15.el9.x86_64 4/6 2023-07-29T13:17:52.774 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:18.1.3-11.g0484097b.el9.x86_64 5/6 2023-07-29T13:17:53.024 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-test 2023-07-29T13:17:53.572 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:39 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:17:53.708 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:17:53.709 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:17:53.709 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-07-29T13:17:53.709 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:17:53.709 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:17:53.710 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:18.1.3-11.g0484097b.el9 ceph 40 M 2023-07-29T13:17:53.710 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-07-29T13:17:53.710 INFO:teuthology.orchestra.run.smithi045.stdout: jq x86_64 1.6-15.el9 CentOS-AppStream 186 k 2023-07-29T13:17:53.710 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:18.1.3-11.g0484097b.el9 ceph 161 k 2023-07-29T13:17:53.710 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-07-29T13:17:53.710 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-07-29T13:17:53.710 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-07-29T13:17:53.710 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:17:53.711 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:17:53.711 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:17:53.711 INFO:teuthology.orchestra.run.smithi045.stdout:Install 6 Packages 2023-07-29T13:17:53.711 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:17:53.711 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 40 M 2023-07-29T13:17:53.712 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 167 M 2023-07-29T13:17:53.712 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:17:53.866 INFO:teuthology.orchestra.run.smithi045.stdout:(1/6): jq-1.6-15.el9.x86_64.rpm 1.2 MB/s | 186 kB 00:00 2023-07-29T13:17:53.924 INFO:teuthology.orchestra.run.smithi045.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.7 MB/s | 217 kB 00:00 2023-07-29T13:17:53.983 INFO:teuthology.orchestra.run.smithi045.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2023-07-29T13:17:54.017 INFO:teuthology.orchestra.run.smithi045.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2023-07-29T13:17:54.117 INFO:teuthology.orchestra.run.smithi045.stdout:(5/6): libcephsqlite-18.1.3-11.g0484097b.el9.x8 397 kB/s | 161 kB 00:00 2023-07-29T13:17:56.142 INFO:teuthology.orchestra.run.smithi045.stdout:(6/6): ceph-test-18.1.3-11.g0484097b.el9.x86_64 16 MB/s | 40 MB 00:02 2023-07-29T13:17:56.145 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:17:56.146 INFO:teuthology.orchestra.run.smithi045.stdout:Total 17 MB/s | 40 MB 00:02 2023-07-29T13:17:56.146 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:17:56.169 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:17:56.169 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:17:56.524 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:17:56.525 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:17:56.726 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:18.1.3-11.g0484097b.el9.x86_64 5/6 2023-07-29T13:17:56.818 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:18.1.3-11.g0484097b.el9.x86_64 6/6 2023-07-29T13:17:57.075 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:17:57.177 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2023-07-29T13:17:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2023-07-29T13:17:57.284 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-07-29T13:17:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : jq-1.6-15.el9.x86_64 4/6 2023-07-29T13:17:57.396 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:18.1.3-11.g0484097b.el9.x86_64 5/6 2023-07-29T13:17:59.135 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-test-2:18.1.3-11.g0484097b.el9.x86_64 6/6 2023-07-29T13:17:59.135 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:18.1.3-11.g0484097b.el9.x86_64 1/6 2023-07-29T13:17:59.136 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:18.1.3-11.g0484097b.el9.x86_64 2/6 2023-07-29T13:17:59.136 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : jq-1.6-15.el9.x86_64 3/6 2023-07-29T13:17:59.136 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2023-07-29T13:17:59.136 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2023-07-29T13:17:59.511 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2023-07-29T13:17:59.512 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:17:59.512 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:17:59.512 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:18.1.3-11.g0484097b.el9.x86_64 jq-1.6-15.el9.x86_64 2023-07-29T13:17:59.512 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:18.1.3-11.g0484097b.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2023-07-29T13:17:59.512 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2023-07-29T13:17:59.513 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:17:59.513 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:17:59.905 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph 2023-07-29T13:18:00.445 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:47 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:00.581 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:00.584 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2023-07-29T13:18:00.584 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-29T13:18:00.584 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2023-07-29T13:18:00.584 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:18:00.584 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:18.1.3-11.g0484097b.el9 ceph 6.5 k 2023-07-29T13:18:00.585 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-29T13:18:00.585 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:18.1.3-11.g0484097b.el9 ceph 2.1 M 2023-07-29T13:18:00.585 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:18.1.3-11.g0484097b.el9 ceph 1.4 M 2023-07-29T13:18:00.585 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 240 k 2023-07-29T13:18:00.585 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:18.1.3-11.g0484097b.el9 ceph 4.4 M 2023-07-29T13:18:00.585 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:18.1.3-11.g0484097b.el9 ceph 18 M 2023-07-29T13:18:00.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-07-29T13:18:00.586 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-07-29T13:18:00.586 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-07-29T13:18:00.586 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-07-29T13:18:00.586 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-07-29T13:18:00.586 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-07-29T13:18:00.586 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-07-29T13:18:00.586 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-07-29T13:18:00.587 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-07-29T13:18:00.587 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-07-29T13:18:00.587 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-07-29T13:18:00.587 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-07-29T13:18:00.587 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-07-29T13:18:00.587 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-07-29T13:18:00.587 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-07-29T13:18:00.587 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-07-29T13:18:00.588 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-07-29T13:18:00.588 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-07-29T13:18:00.588 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-07-29T13:18:00.588 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-07-29T13:18:00.588 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-07-29T13:18:00.588 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-07-29T13:18:00.589 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-07-29T13:18:00.589 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-07-29T13:18:00.589 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-07-29T13:18:00.589 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:00.589 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:18:00.589 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2023-07-29T13:18:00.589 INFO:teuthology.orchestra.run.smithi040.stdout:Install 31 Packages 2023-07-29T13:18:00.589 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:00.590 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 29 M 2023-07-29T13:18:00.590 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 108 M 2023-07-29T13:18:00.590 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:18:00.993 INFO:teuthology.orchestra.run.smithi040.stdout:(1/31): ceph-18.1.3-11.g0484097b.el9.x86_64.rpm 29 kB/s | 6.5 kB 00:00 2023-07-29T13:18:01.318 INFO:teuthology.orchestra.run.smithi040.stdout:(2/31): ceph-mgr-18.1.3-11.g0484097b.el9.x86_64 2.6 MB/s | 1.4 MB 00:00 2023-07-29T13:18:01.369 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:18.1.3-11.g0484097b.el9.x86_64 5/6 2023-07-29T13:18:01.377 INFO:teuthology.orchestra.run.smithi040.stdout:(3/31): ceph-mds-18.1.3-11.g0484097b.el9.x86_64 3.5 MB/s | 2.1 MB 00:00 2023-07-29T13:18:01.463 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:18.1.3-11.g0484097b.el9.x86_64 6/6 2023-07-29T13:18:01.485 INFO:teuthology.orchestra.run.smithi040.stdout:(4/31): ceph-mgr-modules-core-18.1.3-11.g048409 2.2 MB/s | 240 kB 00:00 2023-07-29T13:18:01.611 INFO:teuthology.orchestra.run.smithi040.stdout:(5/31): ceph-mon-18.1.3-11.g0484097b.el9.x86_64 7.1 MB/s | 4.4 MB 00:00 2023-07-29T13:18:01.695 INFO:teuthology.orchestra.run.smithi040.stdout:(6/31): python3-mako-1.1.4-6.el9.noarch.rpm 691 kB/s | 144 kB 00:00 2023-07-29T13:18:01.720 INFO:teuthology.orchestra.run.smithi040.stdout:(7/31): python3-markupsafe-1.1.1-12.el9.x86_64. 294 kB/s | 32 kB 00:00 2023-07-29T13:18:01.745 INFO:teuthology.orchestra.run.smithi040.stdout:(8/31): python3-pytz-2021.1-5.el9.noarch.rpm 944 kB/s | 47 kB 00:00 2023-07-29T13:18:01.770 INFO:teuthology.orchestra.run.smithi040.stdout:(9/31): python3-toml-0.10.2-6.el9.noarch.rpm 767 kB/s | 39 kB 00:00 2023-07-29T13:18:01.804 INFO:teuthology.orchestra.run.smithi040.stdout:(10/31): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2023-07-29T13:18:01.879 INFO:teuthology.orchestra.run.smithi040.stdout:(11/31): python3-cryptography-36.0.1-4.el9.x86_ 11 MB/s | 1.1 MB 00:00 2023-07-29T13:18:01.921 INFO:teuthology.orchestra.run.smithi040.stdout:(12/31): python3-ply-3.11-14.el9.noarch.rpm 887 kB/s | 103 kB 00:00 2023-07-29T13:18:01.955 INFO:teuthology.orchestra.run.smithi040.stdout:(13/31): python3-pycparser-2.20-6.el9.noarch.rp 1.6 MB/s | 124 kB 00:00 2023-07-29T13:18:02.531 INFO:teuthology.orchestra.run.smithi040.stdout:(14/31): ceph-osd-18.1.3-11.g0484097b.el9.x86_6 15 MB/s | 18 MB 00:01 2023-07-29T13:18:02.556 INFO:teuthology.orchestra.run.smithi040.stdout:(15/31): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 68 kB/s | 43 kB 00:00 2023-07-29T13:18:02.773 INFO:teuthology.orchestra.run.smithi040.stdout:(16/31): python3-cheroot-8.6.0-4.el9.noarch.rpm 210 kB/s | 172 kB 00:00 2023-07-29T13:18:03.015 INFO:teuthology.orchestra.run.smithi040.stdout:(17/31): python3-cherrypy-18.6.1-2.el9.noarch.r 742 kB/s | 358 kB 00:00 2023-07-29T13:18:03.040 INFO:teuthology.orchestra.run.smithi040.stdout:(18/31): python3-jaraco-8.2.1-3.el9.noarch.rpm 22 kB/s | 11 kB 00:00 2023-07-29T13:18:03.065 INFO:teuthology.orchestra.run.smithi040.stdout:(19/31): python3-jaraco-classes-3.2.1-5.el9.noa 60 kB/s | 18 kB 00:00 2023-07-29T13:18:03.149 INFO:teuthology.orchestra.run.smithi040.stdout:(20/31): python3-jaraco-collections-3.0.0-8.el9 214 kB/s | 23 kB 00:00 2023-07-29T13:18:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:(21/31): python3-jaraco-functools-3.5.0-2.el9.n 155 kB/s | 19 kB 00:00 2023-07-29T13:18:03.191 INFO:teuthology.orchestra.run.smithi040.stdout:(22/31): python3-jaraco-text-3.2.0-6.el9.noarch 157 kB/s | 20 kB 00:00 2023-07-29T13:18:03.291 INFO:teuthology.orchestra.run.smithi040.stdout:(23/31): python3-logutils-0.3.5-21.el9.noarch.r 326 kB/s | 46 kB 00:00 2023-07-29T13:18:03.325 INFO:teuthology.orchestra.run.smithi040.stdout:(24/31): python3-more-itertools-8.12.0-2.el9.no 496 kB/s | 79 kB 00:00 2023-07-29T13:18:03.375 INFO:teuthology.orchestra.run.smithi040.stdout:(25/31): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2023-07-29T13:18:03.392 INFO:teuthology.orchestra.run.smithi040.stdout:(26/31): python3-portend-3.1.0-2.el9.noarch.rpm 163 kB/s | 16 kB 00:00 2023-07-29T13:18:03.459 INFO:teuthology.orchestra.run.smithi040.stdout:(27/31): python3-pyOpenSSL-21.0.0-1.el9.noarch. 675 kB/s | 90 kB 00:00 2023-07-29T13:18:03.476 INFO:teuthology.orchestra.run.smithi040.stdout:(28/31): python3-tempora-5.0.0-2.el9.noarch.rpm 356 kB/s | 36 kB 00:00 2023-07-29T13:18:03.535 INFO:teuthology.orchestra.run.smithi040.stdout:(29/31): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2023-07-29T13:18:03.626 INFO:teuthology.orchestra.run.smithi040.stdout:(30/31): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2023-07-29T13:18:03.643 INFO:teuthology.orchestra.run.smithi040.stdout:(31/31): python3-zc-lockfile-2.0-10.el9.noarch. 119 kB/s | 20 kB 00:00 2023-07-29T13:18:03.649 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:03.649 INFO:teuthology.orchestra.run.smithi040.stdout:Total 9.6 MB/s | 29 MB 00:03 2023-07-29T13:18:03.649 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:18:03.692 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:18:03.692 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:18:03.896 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-test-2:18.1.3-11.g0484097b.el9.x86_64 6/6 2023-07-29T13:18:03.897 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:18.1.3-11.g0484097b.el9.x86_64 1/6 2023-07-29T13:18:03.897 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:18.1.3-11.g0484097b.el9.x86_64 2/6 2023-07-29T13:18:03.897 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : jq-1.6-15.el9.x86_64 3/6 2023-07-29T13:18:03.897 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2023-07-29T13:18:03.898 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2023-07-29T13:18:04.050 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:18:04.051 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:18:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2023-07-29T13:18:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:04.255 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:18:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:18.1.3-11.g0484097b.el9.x86_64 jq-1.6-15.el9.x86_64 2023-07-29T13:18:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:18.1.3-11.g0484097b.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2023-07-29T13:18:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2023-07-29T13:18:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:04.255 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:04.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph 2023-07-29T13:18:05.106 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:51 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:05.242 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:05.245 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2023-07-29T13:18:05.245 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-07-29T13:18:05.245 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2023-07-29T13:18:05.245 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:18:05.245 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:18.1.3-11.g0484097b.el9 ceph 6.5 k 2023-07-29T13:18:05.245 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-07-29T13:18:05.245 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:18.1.3-11.g0484097b.el9 ceph 2.1 M 2023-07-29T13:18:05.246 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:18.1.3-11.g0484097b.el9 ceph 1.4 M 2023-07-29T13:18:05.246 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 240 k 2023-07-29T13:18:05.246 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:18.1.3-11.g0484097b.el9 ceph 4.4 M 2023-07-29T13:18:05.246 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:18.1.3-11.g0484097b.el9 ceph 18 M 2023-07-29T13:18:05.246 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-07-29T13:18:05.246 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-07-29T13:18:05.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-07-29T13:18:05.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-07-29T13:18:05.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-07-29T13:18:05.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-07-29T13:18:05.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-07-29T13:18:05.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-07-29T13:18:05.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-07-29T13:18:05.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-07-29T13:18:05.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-07-29T13:18:05.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-07-29T13:18:05.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-07-29T13:18:05.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-07-29T13:18:05.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-07-29T13:18:05.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-07-29T13:18:05.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-07-29T13:18:05.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-07-29T13:18:05.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-07-29T13:18:05.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-07-29T13:18:05.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-07-29T13:18:05.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-07-29T13:18:05.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-07-29T13:18:05.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-07-29T13:18:05.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-07-29T13:18:05.250 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:05.250 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:18:05.250 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2023-07-29T13:18:05.250 INFO:teuthology.orchestra.run.smithi045.stdout:Install 31 Packages 2023-07-29T13:18:05.250 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:05.251 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 29 M 2023-07-29T13:18:05.251 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 108 M 2023-07-29T13:18:05.251 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:18:05.255 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:18:05.355 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/31 2023-07-29T13:18:05.395 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/31 2023-07-29T13:18:05.437 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/31 2023-07-29T13:18:05.541 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/31 2023-07-29T13:18:05.625 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/31 2023-07-29T13:18:05.681 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/31 2023-07-29T13:18:05.727 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/31 2023-07-29T13:18:05.786 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 8/31 2023-07-29T13:18:05.876 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 9/31 2023-07-29T13:18:05.995 INFO:teuthology.orchestra.run.smithi045.stdout:(1/31): ceph-18.1.3-11.g0484097b.el9.x86_64.rpm 28 kB/s | 6.5 kB 00:00 2023-07-29T13:18:06.091 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 10/31 2023-07-29T13:18:06.156 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 11/31 2023-07-29T13:18:06.207 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 12/31 2023-07-29T13:18:06.263 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 13/31 2023-07-29T13:18:06.309 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/31 2023-07-29T13:18:06.337 INFO:teuthology.orchestra.run.smithi045.stdout:(2/31): ceph-mds-18.1.3-11.g0484097b.el9.x86_64 3.6 MB/s | 2.1 MB 00:00 2023-07-29T13:18:06.404 INFO:teuthology.orchestra.run.smithi045.stdout:(3/31): ceph-mgr-18.1.3-11.g0484097b.el9.x86_64 2.2 MB/s | 1.4 MB 00:00 2023-07-29T13:18:06.405 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/31 2023-07-29T13:18:06.453 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 16/31 2023-07-29T13:18:06.495 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 17/31 2023-07-29T13:18:06.521 INFO:teuthology.orchestra.run.smithi045.stdout:(4/31): ceph-mgr-modules-core-18.1.3-11.g048409 2.0 MB/s | 240 kB 00:00 2023-07-29T13:18:06.544 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 18/31 2023-07-29T13:18:06.596 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/31 2023-07-29T13:18:06.638 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 20/31 2023-07-29T13:18:06.646 INFO:teuthology.orchestra.run.smithi045.stdout:(5/31): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2023-07-29T13:18:06.688 INFO:teuthology.orchestra.run.smithi045.stdout:(6/31): python3-markupsafe-1.1.1-12.el9.x86_64. 760 kB/s | 32 kB 00:00 2023-07-29T13:18:06.730 INFO:teuthology.orchestra.run.smithi045.stdout:(7/31): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2023-07-29T13:18:06.740 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 21/31 2023-07-29T13:18:06.764 INFO:teuthology.orchestra.run.smithi045.stdout:(8/31): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2023-07-29T13:18:06.794 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 22/31 2023-07-29T13:18:06.822 INFO:teuthology.orchestra.run.smithi045.stdout:(9/31): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2023-07-29T13:18:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/31 2023-07-29T13:18:06.906 INFO:teuthology.orchestra.run.smithi045.stdout:(10/31): python3-cryptography-36.0.1-4.el9.x86_ 14 MB/s | 1.1 MB 00:00 2023-07-29T13:18:06.957 INFO:teuthology.orchestra.run.smithi045.stdout:(11/31): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2023-07-29T13:18:06.998 INFO:teuthology.orchestra.run.smithi045.stdout:(12/31): python3-pycparser-2.20-6.el9.noarch.rp 2.9 MB/s | 124 kB 00:00 2023-07-29T13:18:07.045 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 24/31 2023-07-29T13:18:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 25/31 2023-07-29T13:18:07.316 INFO:teuthology.orchestra.run.smithi045.stdout:(13/31): ceph-osd-18.1.3-11.g0484097b.el9.x86_6 18 MB/s | 18 MB 00:00 2023-07-29T13:18:07.332 INFO:teuthology.orchestra.run.smithi045.stdout:(14/31): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 130 kB/s | 43 kB 00:00 2023-07-29T13:18:07.373 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:18.1.3-11.g0484097b.el9.no 26/31 2023-07-29T13:18:07.408 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:18.1.3-11.g0484097b.el9.x86_64 27/31 2023-07-29T13:18:07.424 INFO:teuthology.orchestra.run.smithi045.stdout:(15/31): ceph-mon-18.1.3-11.g0484097b.el9.x86_6 3.1 MB/s | 4.4 MB 00:01 2023-07-29T13:18:07.433 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:18.1.3-11.g0484097b.el9.x86_64 27/31 2023-07-29T13:18:07.433 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:07.433 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-07-29T13:18:07.434 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-07-29T13:18:07.434 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-07-29T13:18:07.434 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:07.600 INFO:teuthology.orchestra.run.smithi045.stdout:(16/31): python3-jaraco-8.2.1-3.el9.noarch.rpm 61 kB/s | 11 kB 00:00 2023-07-29T13:18:07.708 INFO:teuthology.orchestra.run.smithi045.stdout:(17/31): python3-cherrypy-18.6.1-2.el9.noarch.r 954 kB/s | 358 kB 00:00 2023-07-29T13:18:07.725 INFO:teuthology.orchestra.run.smithi045.stdout:(18/31): python3-jaraco-classes-3.2.1-5.el9.noa 141 kB/s | 18 kB 00:00 2023-07-29T13:18:07.800 INFO:teuthology.orchestra.run.smithi045.stdout:(19/31): python3-cheroot-8.6.0-4.el9.noarch.rpm 354 kB/s | 172 kB 00:00 2023-07-29T13:18:07.826 INFO:teuthology.orchestra.run.smithi045.stdout:(20/31): python3-jaraco-collections-3.0.0-8.el9 198 kB/s | 23 kB 00:00 2023-07-29T13:18:07.851 INFO:teuthology.orchestra.run.smithi045.stdout:(21/31): python3-jaraco-functools-3.5.0-2.el9.n 155 kB/s | 19 kB 00:00 2023-07-29T13:18:07.910 INFO:teuthology.orchestra.run.smithi045.stdout:(22/31): python3-jaraco-text-3.2.0-6.el9.noarch 181 kB/s | 20 kB 00:00 2023-07-29T13:18:07.968 INFO:teuthology.orchestra.run.smithi045.stdout:(23/31): python3-logutils-0.3.5-21.el9.noarch.r 397 kB/s | 46 kB 00:00 2023-07-29T13:18:08.060 INFO:teuthology.orchestra.run.smithi045.stdout:(24/31): python3-more-itertools-8.12.0-2.el9.no 378 kB/s | 79 kB 00:00 2023-07-29T13:18:08.085 INFO:teuthology.orchestra.run.smithi045.stdout:(25/31): python3-portend-3.1.0-2.el9.noarch.rpm 140 kB/s | 16 kB 00:00 2023-07-29T13:18:08.127 INFO:teuthology.orchestra.run.smithi045.stdout:(26/31): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2023-07-29T13:18:08.169 INFO:teuthology.orchestra.run.smithi045.stdout:(27/31): python3-pyOpenSSL-21.0.0-1.el9.noarch. 827 kB/s | 90 kB 00:00 2023-07-29T13:18:08.203 INFO:teuthology.orchestra.run.smithi045.stdout:(28/31): python3-tempora-5.0.0-2.el9.noarch.rpm 304 kB/s | 36 kB 00:00 2023-07-29T13:18:08.262 INFO:teuthology.orchestra.run.smithi045.stdout:(29/31): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2023-07-29T13:18:08.320 INFO:teuthology.orchestra.run.smithi045.stdout:(30/31): python3-zc-lockfile-2.0-10.el9.noarch. 171 kB/s | 20 kB 00:00 2023-07-29T13:18:08.354 INFO:teuthology.orchestra.run.smithi045.stdout:(31/31): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.3 MB/s | 427 kB 00:00 2023-07-29T13:18:08.357 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:08.357 INFO:teuthology.orchestra.run.smithi045.stdout:Total 9.5 MB/s | 29 MB 00:03 2023-07-29T13:18:08.358 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:18:08.399 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:18:08.399 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:18:08.753 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:18:08.753 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:18:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:18.1.3-11.g0484097b.el9.x86_64 28/31 2023-07-29T13:18:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:18.1.3-11.g0484097b.el9.x86_64 28/31 2023-07-29T13:18:09.062 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:09.062 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-07-29T13:18:09.062 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-07-29T13:18:09.062 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-07-29T13:18:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:09.534 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:18.1.3-11.g0484097b.el9.x86_64 29/31 2023-07-29T13:18:09.559 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:18.1.3-11.g0484097b.el9.x86_64 29/31 2023-07-29T13:18:09.559 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:09.559 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-07-29T13:18:09.559 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-07-29T13:18:09.559 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-07-29T13:18:09.560 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:09.776 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:18.1.3-11.g0484097b.el9.x86_64 30/31 2023-07-29T13:18:09.801 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:18.1.3-11.g0484097b.el9.x86_64 30/31 2023-07-29T13:18:09.801 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:09.801 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-07-29T13:18:09.801 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-07-29T13:18:09.801 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-07-29T13:18:09.802 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:09.883 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:18.1.3-11.g0484097b.el9.x86_64 31/31 2023-07-29T13:18:10.031 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:18:10.182 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/31 2023-07-29T13:18:10.231 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/31 2023-07-29T13:18:10.280 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/31 2023-07-29T13:18:10.385 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/31 2023-07-29T13:18:10.469 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/31 2023-07-29T13:18:10.533 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/31 2023-07-29T13:18:10.587 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/31 2023-07-29T13:18:10.645 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 8/31 2023-07-29T13:18:10.718 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 9/31 2023-07-29T13:18:10.931 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 10/31 2023-07-29T13:18:10.992 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 11/31 2023-07-29T13:18:11.048 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 12/31 2023-07-29T13:18:11.098 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 13/31 2023-07-29T13:18:11.136 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/31 2023-07-29T13:18:11.224 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/31 2023-07-29T13:18:11.372 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 16/31 2023-07-29T13:18:11.414 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 17/31 2023-07-29T13:18:11.462 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 18/31 2023-07-29T13:18:11.515 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/31 2023-07-29T13:18:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 20/31 2023-07-29T13:18:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 21/31 2023-07-29T13:18:11.730 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 22/31 2023-07-29T13:18:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-2:18.1.3-11.g0484097b.el9.x86_64 31/31 2023-07-29T13:18:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:18.1.3-11.g0484097b.el9.x86_64 1/31 2023-07-29T13:18:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:18.1.3-11.g0484097b.el9.x86_64 2/31 2023-07-29T13:18:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:18.1.3-11.g0484097b.el9.x86_64 3/31 2023-07-29T13:18:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:18.1.3-11.g0484097b.el9.x86_64 4/31 2023-07-29T13:18:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:18.1.3-11.g0484097b.el9.x86_64 5/31 2023-07-29T13:18:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:18.1.3-11.g0484097b.el9.no 6/31 2023-07-29T13:18:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/31 2023-07-29T13:18:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/31 2023-07-29T13:18:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/31 2023-07-29T13:18:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/31 2023-07-29T13:18:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/31 2023-07-29T13:18:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/31 2023-07-29T13:18:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/31 2023-07-29T13:18:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/31 2023-07-29T13:18:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 15/31 2023-07-29T13:18:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 16/31 2023-07-29T13:18:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 17/31 2023-07-29T13:18:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 18/31 2023-07-29T13:18:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 19/31 2023-07-29T13:18:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 20/31 2023-07-29T13:18:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 21/31 2023-07-29T13:18:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 22/31 2023-07-29T13:18:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 23/31 2023-07-29T13:18:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 24/31 2023-07-29T13:18:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 25/31 2023-07-29T13:18:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 26/31 2023-07-29T13:18:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/31 2023-07-29T13:18:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 28/31 2023-07-29T13:18:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 29/31 2023-07-29T13:18:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 30/31 2023-07-29T13:18:11.812 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/31 2023-07-29T13:18:11.972 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 24/31 2023-07-29T13:18:12.101 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 25/31 2023-07-29T13:18:12.274 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:18.1.3-11.g0484097b.el9.no 26/31 2023-07-29T13:18:12.301 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:18.1.3-11.g0484097b.el9.x86_64 27/31 2023-07-29T13:18:12.327 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:18.1.3-11.g0484097b.el9.x86_64 27/31 2023-07-29T13:18:12.327 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:12.327 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-07-29T13:18:12.327 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-07-29T13:18:12.328 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-07-29T13:18:12.328 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 31/31 2023-07-29T13:18:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:12.634 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:18:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-07-29T13:18:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-07-29T13:18:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-07-29T13:18:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-07-29T13:18:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-07-29T13:18:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-07-29T13:18:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-07-29T13:18:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-07-29T13:18:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-07-29T13:18:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-07-29T13:18:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-07-29T13:18:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2023-07-29T13:18:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-07-29T13:18:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-07-29T13:18:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-07-29T13:18:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2023-07-29T13:18:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2023-07-29T13:18:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-07-29T13:18:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2023-07-29T13:18:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2023-07-29T13:18:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-07-29T13:18:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2023-07-29T13:18:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.7-6.el9.noarch 2023-07-29T13:18:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-07-29T13:18:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-07-29T13:18:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:12.639 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:13.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-base 2023-07-29T13:18:13.597 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:00 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:13.679 INFO:teuthology.orchestra.run.smithi040.stdout:Package ceph-base-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:18:13.731 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:13.732 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2023-07-29T13:18:13.732 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:13.795 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install cephadm 2023-07-29T13:18:13.927 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:18.1.3-11.g0484097b.el9.x86_64 28/31 2023-07-29T13:18:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:18.1.3-11.g0484097b.el9.x86_64 28/31 2023-07-29T13:18:13.955 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:13.955 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-07-29T13:18:13.955 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-07-29T13:18:13.955 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-07-29T13:18:13.955 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:14.339 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:01 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:14.444 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:18.1.3-11.g0484097b.el9.x86_64 29/31 2023-07-29T13:18:14.468 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:18.1.3-11.g0484097b.el9.x86_64 29/31 2023-07-29T13:18:14.469 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:14.469 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-07-29T13:18:14.469 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-07-29T13:18:14.469 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-07-29T13:18:14.469 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:14.472 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:14.473 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:14.473 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-29T13:18:14.473 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:14.473 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:18:14.474 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 209 k 2023-07-29T13:18:14.474 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:14.474 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:18:14.474 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:14.474 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:18:14.474 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:14.475 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 209 k 2023-07-29T13:18:14.475 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 200 k 2023-07-29T13:18:14.475 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:18:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:18.1.3-11.g0484097b.el9.x86_64 30/31 2023-07-29T13:18:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:18.1.3-11.g0484097b.el9.x86_64 30/31 2023-07-29T13:18:14.720 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:14.720 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-07-29T13:18:14.720 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-07-29T13:18:14.721 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-07-29T13:18:14.721 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:14.794 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:18.1.3-11.g0484097b.el9.x86_64 31/31 2023-07-29T13:18:14.842 INFO:teuthology.orchestra.run.smithi040.stdout:cephadm-18.1.3-11.g0484097b.el9.noarch.rpm 567 kB/s | 209 kB 00:00 2023-07-29T13:18:14.843 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:14.843 INFO:teuthology.orchestra.run.smithi040.stdout:Total 565 kB/s | 209 kB 00:00 2023-07-29T13:18:14.844 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:18:14.849 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:18:14.849 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:18:14.851 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:18:14.852 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:18:14.925 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:18:15.379 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:15.490 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:15.864 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:16.152 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:16.153 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:16.153 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:18:16.153 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:16.153 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:16.153 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:16.381 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-immutable-object-cache 2023-07-29T13:18:16.809 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-2:18.1.3-11.g0484097b.el9.x86_64 31/31 2023-07-29T13:18:16.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:18.1.3-11.g0484097b.el9.x86_64 1/31 2023-07-29T13:18:16.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:18.1.3-11.g0484097b.el9.x86_64 2/31 2023-07-29T13:18:16.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:18.1.3-11.g0484097b.el9.x86_64 3/31 2023-07-29T13:18:16.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:18.1.3-11.g0484097b.el9.x86_64 4/31 2023-07-29T13:18:16.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:18.1.3-11.g0484097b.el9.x86_64 5/31 2023-07-29T13:18:16.811 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:18.1.3-11.g0484097b.el9.no 6/31 2023-07-29T13:18:16.811 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/31 2023-07-29T13:18:16.811 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/31 2023-07-29T13:18:16.811 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/31 2023-07-29T13:18:16.811 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/31 2023-07-29T13:18:16.811 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/31 2023-07-29T13:18:16.811 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/31 2023-07-29T13:18:16.812 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/31 2023-07-29T13:18:16.812 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/31 2023-07-29T13:18:16.812 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 15/31 2023-07-29T13:18:16.812 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 16/31 2023-07-29T13:18:16.812 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 17/31 2023-07-29T13:18:16.812 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 18/31 2023-07-29T13:18:16.812 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 19/31 2023-07-29T13:18:16.813 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 20/31 2023-07-29T13:18:16.813 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 21/31 2023-07-29T13:18:16.813 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 22/31 2023-07-29T13:18:16.813 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 23/31 2023-07-29T13:18:16.813 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 24/31 2023-07-29T13:18:16.813 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 25/31 2023-07-29T13:18:16.813 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 26/31 2023-07-29T13:18:16.814 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/31 2023-07-29T13:18:16.814 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 28/31 2023-07-29T13:18:16.814 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 29/31 2023-07-29T13:18:16.814 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 30/31 2023-07-29T13:18:16.921 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:03 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:17.058 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:17.059 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:17.059 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:18:17.059 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:17.059 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:18:17.059 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:18.1.3-11.g0484097b.el9 ceph 142 k 2023-07-29T13:18:17.059 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:17.060 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:18:17.060 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:17.060 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:18:17.060 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:17.060 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 142 k 2023-07-29T13:18:17.060 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 400 k 2023-07-29T13:18:17.061 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:18:17.407 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-immutable-object-cache-18.1.3-11.g0484097b 410 kB/s | 142 kB 00:00 2023-07-29T13:18:17.408 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:17.408 INFO:teuthology.orchestra.run.smithi040.stdout:Total 408 kB/s | 142 kB 00:00 2023-07-29T13:18:17.409 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:18:17.417 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:18:17.418 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:18:17.477 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:18:17.477 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:18:17.624 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:18:17.692 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:18.1.3-11.g0484097b.el 1/1 2023-07-29T13:18:17.717 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.1.3-11.g0484097b.el 1/1 2023-07-29T13:18:17.717 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:17.717 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-07-29T13:18:17.717 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:18.012 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 31/31 2023-07-29T13:18:18.012 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:18.012 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:18:18.012 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:18.012 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:18.012 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:18.013 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:18.013 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:18.013 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:18.013 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-07-29T13:18:18.013 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-07-29T13:18:18.013 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-07-29T13:18:18.013 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-07-29T13:18:18.014 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-07-29T13:18:18.014 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-07-29T13:18:18.014 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-07-29T13:18:18.014 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-07-29T13:18:18.014 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-07-29T13:18:18.014 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-07-29T13:18:18.014 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-07-29T13:18:18.014 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2023-07-29T13:18:18.015 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-07-29T13:18:18.015 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-07-29T13:18:18.015 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-07-29T13:18:18.015 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2023-07-29T13:18:18.015 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2023-07-29T13:18:18.015 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-07-29T13:18:18.015 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2023-07-29T13:18:18.016 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2023-07-29T13:18:18.016 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-07-29T13:18:18.016 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2023-07-29T13:18:18.016 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2023-07-29T13:18:18.016 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-07-29T13:18:18.016 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-07-29T13:18:18.016 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:18.016 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:18.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-base 2023-07-29T13:18:18.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:18.1.3-11.g0484097b.el 1/1 2023-07-29T13:18:18.752 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:18.752 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:18:18.753 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:18.753 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:18.753 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:18.955 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:04 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:18.994 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr 2023-07-29T13:18:19.039 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-base-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:18:19.091 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:19.092 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-07-29T13:18:19.093 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:19.164 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install cephadm 2023-07-29T13:18:19.540 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:06 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:19.622 INFO:teuthology.orchestra.run.smithi040.stdout:Package ceph-mgr-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:18:19.674 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:19.675 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2023-07-29T13:18:19.675 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:19.706 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:05 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:19.769 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-dashboard 2023-07-29T13:18:19.841 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:19.842 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:19.842 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-07-29T13:18:19.842 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:19.843 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:18:19.843 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 209 k 2023-07-29T13:18:19.843 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:19.843 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:18:19.843 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:19.843 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:18:19.843 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:19.844 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 209 k 2023-07-29T13:18:19.844 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 200 k 2023-07-29T13:18:19.844 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:18:20.227 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm-18.1.3-11.g0484097b.el9.noarch.rpm 544 kB/s | 209 kB 00:00 2023-07-29T13:18:20.228 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:20.228 INFO:teuthology.orchestra.run.smithi045.stdout:Total 542 kB/s | 209 kB 00:00 2023-07-29T13:18:20.229 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:18:20.234 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:18:20.234 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:18:20.237 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:18:20.237 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:18:20.307 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:18:20.315 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:07 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:20.451 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:20.452 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:20.452 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-29T13:18:20.452 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:20.452 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:18:20.452 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 1.7 M 2023-07-29T13:18:20.452 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-29T13:18:20.453 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 23 k 2023-07-29T13:18:20.453 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 15 k 2023-07-29T13:18:20.453 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-07-29T13:18:20.453 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-07-29T13:18:20.453 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-07-29T13:18:20.453 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2023-07-29T13:18:20.453 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-07-29T13:18:20.453 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:20.454 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:18:20.454 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:20.454 INFO:teuthology.orchestra.run.smithi040.stdout:Install 7 Packages 2023-07-29T13:18:20.454 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:20.454 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 2.0 M 2023-07-29T13:18:20.455 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 8.1 M 2023-07-29T13:18:20.455 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:18:20.797 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:20.901 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:21.121 INFO:teuthology.orchestra.run.smithi040.stdout:(1/7): ceph-grafana-dashboards-18.1.3-11.g04840 88 kB/s | 23 kB 00:00 2023-07-29T13:18:21.146 INFO:teuthology.orchestra.run.smithi040.stdout:(2/7): ceph-prometheus-alerts-18.1.3-11.g048409 51 kB/s | 15 kB 00:00 2023-07-29T13:18:21.241 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:21.254 INFO:teuthology.orchestra.run.smithi040.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 68 kB/s | 9.0 kB 00:00 2023-07-29T13:18:21.313 INFO:teuthology.orchestra.run.smithi040.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 245 kB/s | 41 kB 00:00 2023-07-29T13:18:21.372 INFO:teuthology.orchestra.run.smithi040.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 263 kB/s | 31 kB 00:00 2023-07-29T13:18:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:(6/7): ceph-mgr-dashboard-18.1.3-11.g0484097b.e 3.1 MB/s | 1.7 MB 00:00 2023-07-29T13:18:21.464 INFO:teuthology.orchestra.run.smithi040.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.2 MB/s | 188 kB 00:00 2023-07-29T13:18:21.466 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:21.466 INFO:teuthology.orchestra.run.smithi040.stdout:Total 2.0 MB/s | 2.0 MB 00:01 2023-07-29T13:18:21.466 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:18:21.475 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:18:21.475 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:18:21.535 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:18:21.535 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:18:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:21.553 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:18:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:21.554 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:21.554 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:21.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-immutable-object-cache 2023-07-29T13:18:22.032 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:18:22.090 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-07-29T13:18:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-07-29T13:18:22.172 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-07-29T13:18:22.203 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-07-29T13:18:22.232 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:18.1.3-11.g0484097b.el9.noa 5/7 2023-07-29T13:18:22.344 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:08 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:22.464 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:18.1.3-11.g0484097b.el9.no 6/7 2023-07-29T13:18:22.478 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:22.479 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:18:22.480 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:22.480 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:18:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:18.1.3-11.g0484097b.el9 ceph 142 k 2023-07-29T13:18:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:22.480 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:18:22.480 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:22.481 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:18:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:22.481 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 142 k 2023-07-29T13:18:22.481 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 400 k 2023-07-29T13:18:22.481 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:18:22.501 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:18.1.3-11.g0484097b.el9.noarch 7/7 2023-07-29T13:18:22.817 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-immutable-object-cache-18.1.3-11.g0484097b 423 kB/s | 142 kB 00:00 2023-07-29T13:18:22.818 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:22.818 INFO:teuthology.orchestra.run.smithi045.stdout:Total 421 kB/s | 142 kB 00:00 2023-07-29T13:18:22.819 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:18:22.827 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:18:22.827 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:18:22.886 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:18:22.887 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:18:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:18:23.073 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.1.3-11.g0484097b.el9.noarch 7/7 2023-07-29T13:18:23.073 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:18.1.3-11.g0484097b.el9.no 1/7 2023-07-29T13:18:23.073 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:18.1.3-11.g0484097b.el9.noarch 2/7 2023-07-29T13:18:23.073 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:18.1.3-11.g0484097b.el9.noa 3/7 2023-07-29T13:18:23.074 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-07-29T13:18:23.074 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-07-29T13:18:23.074 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-07-29T13:18:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:18.1.3-11.g0484097b.el 1/1 2023-07-29T13:18:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.1.3-11.g0484097b.el 1/1 2023-07-29T13:18:23.136 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:23.136 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-07-29T13:18:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:23.386 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-07-29T13:18:23.387 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:23.387 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:18:23.387 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:23.387 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:23.387 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:23.387 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-07-29T13:18:23.388 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-07-29T13:18:23.388 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-07-29T13:18:23.388 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2023-07-29T13:18:23.388 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:23.388 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:23.607 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-diskprediction-local 2023-07-29T13:18:24.151 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:11 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:24.169 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:18.1.3-11.g0484097b.el 1/1 2023-07-29T13:18:24.169 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:24.169 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:18:24.170 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:24.170 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:24.170 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:24.288 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:24.289 INFO:teuthology.orchestra.run.smithi040.stdout:========================================================================================== 2023-07-29T13:18:24.289 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-29T13:18:24.290 INFO:teuthology.orchestra.run.smithi040.stdout:========================================================================================== 2023-07-29T13:18:24.290 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:18:24.290 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 7.4 M 2023-07-29T13:18:24.290 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-29T13:18:24.290 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-07-29T13:18:24.290 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-07-29T13:18:24.290 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-07-29T13:18:24.291 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.4.1-2.1.el9 CentOS-BaseOS 807 k 2023-07-29T13:18:24.291 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.4.1-2.1.el9 CentOS-BaseOS 197 k 2023-07-29T13:18:24.291 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-07-29T13:18:24.291 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-07-29T13:18:24.291 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.16-1.el9 appstream 247 k 2023-07-29T13:18:24.291 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-07-29T13:18:24.291 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-07-29T13:18:24.292 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-07-29T13:18:24.292 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:24.292 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:18:24.292 INFO:teuthology.orchestra.run.smithi040.stdout:========================================================================================== 2023-07-29T13:18:24.292 INFO:teuthology.orchestra.run.smithi040.stdout:Install 12 Packages 2023-07-29T13:18:24.292 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:24.293 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 37 M 2023-07-29T13:18:24.293 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 209 M 2023-07-29T13:18:24.293 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:18:24.418 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr 2023-07-29T13:18:24.538 INFO:teuthology.orchestra.run.smithi040.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 205 kB/s | 32 kB 00:00 2023-07-29T13:18:24.580 INFO:teuthology.orchestra.run.smithi040.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 422 kB/s | 17 kB 00:00 2023-07-29T13:18:24.622 INFO:teuthology.orchestra.run.smithi040.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 835 kB/s | 34 kB 00:00 2023-07-29T13:18:24.689 INFO:teuthology.orchestra.run.smithi040.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.8 MB/s | 3.0 MB 00:00 2023-07-29T13:18:24.847 INFO:teuthology.orchestra.run.smithi040.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-07-29T13:18:24.922 INFO:teuthology.orchestra.run.smithi040.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2023-07-29T13:18:24.948 INFO:teuthology.orchestra.run.smithi040.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.2 MB/s | 217 kB 00:00 2023-07-29T13:18:24.963 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:10 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:25.048 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-mgr-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:18:25.100 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:25.101 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-07-29T13:18:25.101 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:25.279 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-dashboard 2023-07-29T13:18:25.315 INFO:teuthology.orchestra.run.smithi040.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2023-07-29T13:18:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:(9/12): libgfortran-11.4.1-2.1.el9.x86_64.rpm 2.0 MB/s | 807 kB 00:00 2023-07-29T13:18:25.374 INFO:teuthology.orchestra.run.smithi040.stdout:(10/12): libquadmath-11.4.1-2.1.el9.x86_64.rpm 3.4 MB/s | 197 kB 00:00 2023-07-29T13:18:25.574 INFO:teuthology.orchestra.run.smithi040.stdout:(11/12): ceph-mgr-diskprediction-local-18.1.3-1 6.2 MB/s | 7.4 MB 00:01 2023-07-29T13:18:25.815 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:11 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:25.924 INFO:teuthology.orchestra.run.smithi040.stdout:(12/12): python3-devel-3.9.16-1.el9.x86_64.rpm 430 kB/s | 247 kB 00:00 2023-07-29T13:18:25.927 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:25.928 INFO:teuthology.orchestra.run.smithi040.stdout:Total 23 MB/s | 37 MB 00:01 2023-07-29T13:18:25.951 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:18:25.953 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:25.954 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:25.954 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-07-29T13:18:25.954 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:25.954 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:18:25.955 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 1.7 M 2023-07-29T13:18:25.955 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-07-29T13:18:25.955 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 23 k 2023-07-29T13:18:25.955 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 15 k 2023-07-29T13:18:25.955 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-07-29T13:18:25.955 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-07-29T13:18:25.955 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-07-29T13:18:25.956 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2023-07-29T13:18:25.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-07-29T13:18:25.956 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:25.956 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:18:25.956 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:25.956 INFO:teuthology.orchestra.run.smithi045.stdout:Install 7 Packages 2023-07-29T13:18:25.956 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:25.957 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 2.0 M 2023-07-29T13:18:25.957 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 8.1 M 2023-07-29T13:18:25.957 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:18:25.980 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:18:25.980 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:18:26.428 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:18:26.429 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:18:26.630 INFO:teuthology.orchestra.run.smithi045.stdout:(1/7): ceph-grafana-dashboards-18.1.3-11.g04840 88 kB/s | 23 kB 00:00 2023-07-29T13:18:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:(2/7): ceph-prometheus-alerts-18.1.3-11.g048409 43 kB/s | 15 kB 00:00 2023-07-29T13:18:26.888 INFO:teuthology.orchestra.run.smithi045.stdout:(3/7): ceph-mgr-dashboard-18.1.3-11.g0484097b.e 3.3 MB/s | 1.7 MB 00:00 2023-07-29T13:18:27.039 INFO:teuthology.orchestra.run.smithi045.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 22 kB/s | 9.0 kB 00:00 2023-07-29T13:18:27.154 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:18:27.216 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 1/12 2023-07-29T13:18:27.245 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 2/12 2023-07-29T13:18:27.289 INFO:teuthology.orchestra.run.smithi045.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 70 kB/s | 41 kB 00:00 2023-07-29T13:18:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-07-29T13:18:27.397 INFO:teuthology.orchestra.run.smithi045.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 60 kB/s | 31 kB 00:00 2023-07-29T13:18:27.398 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.16-1.el9.x86_64 4/12 2023-07-29T13:18:27.422 INFO:teuthology.orchestra.run.smithi045.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 491 kB/s | 188 kB 00:00 2023-07-29T13:18:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.4 MB/s | 2.0 MB 00:01 2023-07-29T13:18:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:18:27.436 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:18:27.436 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:18:27.496 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:18:27.496 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:18:27.583 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-07-29T13:18:27.614 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-07-29T13:18:27.697 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-07-29T13:18:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:18:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-07-29T13:18:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-07-29T13:18:28.022 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-07-29T13:18:28.053 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-07-29T13:18:28.082 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:18.1.3-11.g0484097b.el9.noa 5/7 2023-07-29T13:18:28.322 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:18.1.3-11.g0484097b.el9.no 6/7 2023-07-29T13:18:28.351 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:18.1.3-11.g0484097b.el9.noarch 7/7 2023-07-29T13:18:28.392 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-07-29T13:18:28.515 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-07-29T13:18:28.844 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.1.3-11.g0484097b.el9.noarch 7/7 2023-07-29T13:18:28.844 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:18.1.3-11.g0484097b.el9.no 1/7 2023-07-29T13:18:28.844 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:18.1.3-11.g0484097b.el9.noarch 2/7 2023-07-29T13:18:28.844 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:18.1.3-11.g0484097b.el9.noa 3/7 2023-07-29T13:18:28.844 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-07-29T13:18:28.845 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-07-29T13:18:28.845 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-07-29T13:18:29.160 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-07-29T13:18:29.161 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:29.161 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:18:29.161 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:29.161 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:29.161 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:29.161 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-07-29T13:18:29.161 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-07-29T13:18:29.162 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-07-29T13:18:29.162 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2023-07-29T13:18:29.162 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:29.162 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:29.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-diskprediction-local 2023-07-29T13:18:29.841 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-07-29T13:18:29.920 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:15 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:30.057 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:30.058 INFO:teuthology.orchestra.run.smithi045.stdout:========================================================================================== 2023-07-29T13:18:30.058 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-07-29T13:18:30.058 INFO:teuthology.orchestra.run.smithi045.stdout:========================================================================================== 2023-07-29T13:18:30.058 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:18:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 7.4 M 2023-07-29T13:18:30.059 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-07-29T13:18:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-07-29T13:18:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-07-29T13:18:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-07-29T13:18:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-2.1.el9 CentOS-BaseOS 807 k 2023-07-29T13:18:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-2.1.el9 CentOS-BaseOS 197 k 2023-07-29T13:18:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-07-29T13:18:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-07-29T13:18:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.16-1.el9 appstream 247 k 2023-07-29T13:18:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-07-29T13:18:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-07-29T13:18:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-07-29T13:18:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:30.061 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:18:30.061 INFO:teuthology.orchestra.run.smithi045.stdout:========================================================================================== 2023-07-29T13:18:30.061 INFO:teuthology.orchestra.run.smithi045.stdout:Install 12 Packages 2023-07-29T13:18:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:30.062 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 37 M 2023-07-29T13:18:30.062 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 209 M 2023-07-29T13:18:30.062 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:18:30.330 INFO:teuthology.orchestra.run.smithi045.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 234 kB/s | 32 kB 00:00 2023-07-29T13:18:30.438 INFO:teuthology.orchestra.run.smithi045.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 159 kB/s | 17 kB 00:00 2023-07-29T13:18:30.489 INFO:teuthology.orchestra.run.smithi045.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 697 kB/s | 34 kB 00:00 2023-07-29T13:18:30.606 INFO:teuthology.orchestra.run.smithi045.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.3 MB/s | 3.0 MB 00:00 2023-07-29T13:18:30.698 INFO:teuthology.orchestra.run.smithi045.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2023-07-29T13:18:30.798 INFO:teuthology.orchestra.run.smithi045.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2023-07-29T13:18:30.823 INFO:teuthology.orchestra.run.smithi045.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 217 kB 00:00 2023-07-29T13:18:30.926 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-07-29T13:18:30.955 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:18.1.3-11.g0484097 12/12 2023-07-29T13:18:31.031 INFO:teuthology.orchestra.run.smithi045.stdout:(8/12): libgfortran-11.4.1-2.1.el9.x86_64.rpm 3.8 MB/s | 807 kB 00:00 2023-07-29T13:18:31.249 INFO:teuthology.orchestra.run.smithi045.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2023-07-29T13:18:31.274 INFO:teuthology.orchestra.run.smithi045.stdout:(10/12): libquadmath-11.4.1-2.1.el9.x86_64.rpm 825 kB/s | 197 kB 00:00 2023-07-29T13:18:31.891 INFO:teuthology.orchestra.run.smithi045.stdout:(11/12): python3-devel-3.9.16-1.el9.x86_64.rpm 401 kB/s | 247 kB 00:00 2023-07-29T13:18:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:(12/12): ceph-mgr-diskprediction-local-18.1.3-1 4.1 MB/s | 7.4 MB 00:01 2023-07-29T13:18:32.028 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:32.029 INFO:teuthology.orchestra.run.smithi045.stdout:Total 19 MB/s | 37 MB 00:01 2023-07-29T13:18:32.052 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:18:32.080 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:18:32.080 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:18:32.515 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:18:32.516 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:18:33.236 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:18:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 1/12 2023-07-29T13:18:33.362 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 2/12 2023-07-29T13:18:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-07-29T13:18:33.508 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.16-1.el9.x86_64 4/12 2023-07-29T13:18:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.1.3-11.g0484097 12/12 2023-07-29T13:18:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.1.3-11.g0484097 1/12 2023-07-29T13:18:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-07-29T13:18:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-07-29T13:18:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-07-29T13:18:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-07-29T13:18:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-07-29T13:18:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2023-07-29T13:18:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2023-07-29T13:18:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2023-07-29T13:18:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 10/12 2023-07-29T13:18:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 11/12 2023-07-29T13:18:33.693 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-07-29T13:18:33.723 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-07-29T13:18:33.807 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-07-29T13:18:33.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.16-1.el9.x86_64 12/12 2023-07-29T13:18:33.994 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:33.994 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:18:33.994 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:33.994 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-07-29T13:18:33.994 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-07-29T13:18:33.994 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-07-29T13:18:33.994 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-07-29T13:18:33.995 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-07-29T13:18:33.995 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.21-2.el9.x86_64 2023-07-29T13:18:33.995 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-07-29T13:18:33.995 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.16-1.el9.x86_64 2023-07-29T13:18:33.995 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-07-29T13:18:33.995 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-07-29T13:18:33.995 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-07-29T13:18:33.995 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:33.996 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:34.426 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-rook 2023-07-29T13:18:34.493 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-07-29T13:18:34.632 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-07-29T13:18:34.975 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:21 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:35.112 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:35.113 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2023-07-29T13:18:35.113 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-29T13:18:35.113 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2023-07-29T13:18:35.113 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:18:35.113 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 47 k 2023-07-29T13:18:35.114 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-29T13:18:35.114 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-07-29T13:18:35.114 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-07-29T13:18:35.114 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-07-29T13:18:35.114 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-07-29T13:18:35.114 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-07-29T13:18:35.114 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-07-29T13:18:35.114 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-07-29T13:18:35.115 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-07-29T13:18:35.115 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-07-29T13:18:35.115 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-07-29T13:18:35.115 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-07-29T13:18:35.115 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-07-29T13:18:35.115 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:35.115 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:18:35.116 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2023-07-29T13:18:35.116 INFO:teuthology.orchestra.run.smithi040.stdout:Install 13 Packages 2023-07-29T13:18:35.116 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:35.116 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 2.0 M 2023-07-29T13:18:35.117 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 26 M 2023-07-29T13:18:35.117 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:18:35.415 INFO:teuthology.orchestra.run.smithi040.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 180 kB/s | 24 kB 00:00 2023-07-29T13:18:35.440 INFO:teuthology.orchestra.run.smithi040.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 112 kB/s | 18 kB 00:00 2023-07-29T13:18:35.499 INFO:teuthology.orchestra.run.smithi040.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 174 kB 00:00 2023-07-29T13:18:35.541 INFO:teuthology.orchestra.run.smithi040.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2023-07-29T13:18:35.575 INFO:teuthology.orchestra.run.smithi040.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 211 kB 00:00 2023-07-29T13:18:35.600 INFO:teuthology.orchestra.run.smithi040.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 733 kB/s | 43 kB 00:00 2023-07-29T13:18:35.625 INFO:teuthology.orchestra.run.smithi040.stdout:(7/13): ceph-mgr-rook-18.1.3-11.g0484097b.el9.n 137 kB/s | 47 kB 00:00 2023-07-29T13:18:35.834 INFO:teuthology.orchestra.run.smithi040.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 124 kB/s | 32 kB 00:00 2023-07-29T13:18:35.859 INFO:teuthology.orchestra.run.smithi040.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 58 kB/s | 15 kB 00:00 2023-07-29T13:18:35.951 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-07-29T13:18:36.009 INFO:teuthology.orchestra.run.smithi040.stdout:(10/13): python3-google-auth-2.22.0-1.el9.noarc 512 kB/s | 197 kB 00:00 2023-07-29T13:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 3.4 MB/s | 1.0 MB 00:00 2023-07-29T13:18:36.160 INFO:teuthology.orchestra.run.smithi040.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 197 kB/s | 59 kB 00:00 2023-07-29T13:18:36.193 INFO:teuthology.orchestra.run.smithi040.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 488 kB/s | 90 kB 00:00 2023-07-29T13:18:36.198 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:36.198 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2023-07-29T13:18:36.199 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:18:36.213 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:18:36.214 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:18:36.309 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:18:36.310 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:18:36.862 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:18:37.030 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-07-29T13:18:37.064 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:18.1.3-11.g0484097 12/12 2023-07-29T13:18:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-07-29T13:18:37.138 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-07-29T13:18:37.211 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-07-29T13:18:37.263 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-07-29T13:18:37.313 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-07-29T13:18:37.417 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-07-29T13:18:37.545 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 7/13 2023-07-29T13:18:37.615 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-07-29T13:18:38.398 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-07-29T13:18:38.470 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-07-29T13:18:38.512 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-07-29T13:18:38.567 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-07-29T13:18:38.598 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:18.1.3-11.g0484097b.el9.noarch 13/13 2023-07-29T13:18:39.383 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:18.1.3-11.g0484097b.el9.noarch 13/13 2023-07-29T13:18:39.383 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:18.1.3-11.g0484097b.el9.noarch 1/13 2023-07-29T13:18:39.384 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-07-29T13:18:39.384 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-07-29T13:18:39.384 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-07-29T13:18:39.384 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-07-29T13:18:39.385 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-07-29T13:18:39.385 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-07-29T13:18:39.385 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-07-29T13:18:39.386 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-07-29T13:18:39.386 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 10/13 2023-07-29T13:18:39.386 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-07-29T13:18:39.386 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-07-29T13:18:39.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-07-29T13:18:39.752 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:39.752 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:18:39.752 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:39.752 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-07-29T13:18:39.752 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-07-29T13:18:39.752 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-07-29T13:18:39.753 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-07-29T13:18:39.753 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-07-29T13:18:39.753 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-07-29T13:18:39.753 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-07-29T13:18:39.753 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-07-29T13:18:39.753 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-07-29T13:18:39.753 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-07-29T13:18:39.753 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2023-07-29T13:18:39.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-07-29T13:18:39.754 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:39.754 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:39.765 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.1.3-11.g0484097 12/12 2023-07-29T13:18:39.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.1.3-11.g0484097 1/12 2023-07-29T13:18:39.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-07-29T13:18:39.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-07-29T13:18:39.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-07-29T13:18:39.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-07-29T13:18:39.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-07-29T13:18:39.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2023-07-29T13:18:39.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2023-07-29T13:18:39.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2023-07-29T13:18:39.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 10/12 2023-07-29T13:18:39.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 11/12 2023-07-29T13:18:40.016 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-cephadm 2023-07-29T13:18:40.179 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.16-1.el9.x86_64 12/12 2023-07-29T13:18:40.179 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:40.179 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:18:40.179 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:40.180 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-07-29T13:18:40.180 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-07-29T13:18:40.180 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-07-29T13:18:40.180 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-07-29T13:18:40.180 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-07-29T13:18:40.180 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2023-07-29T13:18:40.181 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-07-29T13:18:40.181 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.16-1.el9.x86_64 2023-07-29T13:18:40.181 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-07-29T13:18:40.181 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-07-29T13:18:40.181 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-07-29T13:18:40.182 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:40.182 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:40.561 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:27 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:40.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-rook 2023-07-29T13:18:40.695 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:40.696 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2023-07-29T13:18:40.696 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-29T13:18:40.696 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2023-07-29T13:18:40.696 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:18:40.696 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 127 k 2023-07-29T13:18:40.696 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-29T13:18:40.696 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-07-29T13:18:40.697 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-07-29T13:18:40.697 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-07-29T13:18:40.697 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-07-29T13:18:40.697 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-07-29T13:18:40.697 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:40.697 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:18:40.697 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2023-07-29T13:18:40.698 INFO:teuthology.orchestra.run.smithi040.stdout:Install 6 Packages 2023-07-29T13:18:40.698 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:40.698 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 6.8 M 2023-07-29T13:18:40.698 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 33 M 2023-07-29T13:18:40.699 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:18:40.954 INFO:teuthology.orchestra.run.smithi040.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.2 MB/s | 228 kB 00:00 2023-07-29T13:18:41.129 INFO:teuthology.orchestra.run.smithi040.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2023-07-29T13:18:41.205 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:27 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:41.238 INFO:teuthology.orchestra.run.smithi040.stdout:(3/6): ceph-mgr-cephadm-18.1.3-11.g0484097b.el9 268 kB/s | 127 kB 00:00 2023-07-29T13:18:41.339 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:41.341 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2023-07-29T13:18:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-07-29T13:18:41.341 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2023-07-29T13:18:41.341 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:18:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 47 k 2023-07-29T13:18:41.342 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-07-29T13:18:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-07-29T13:18:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-07-29T13:18:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-07-29T13:18:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-07-29T13:18:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-07-29T13:18:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-07-29T13:18:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-07-29T13:18:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-07-29T13:18:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-07-29T13:18:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-07-29T13:18:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-07-29T13:18:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-07-29T13:18:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:41.344 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:18:41.344 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2023-07-29T13:18:41.344 INFO:teuthology.orchestra.run.smithi045.stdout:Install 13 Packages 2023-07-29T13:18:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:41.345 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 2.0 M 2023-07-29T13:18:41.345 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 26 M 2023-07-29T13:18:41.345 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:18:41.540 INFO:teuthology.orchestra.run.smithi045.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 198 kB/s | 24 kB 00:00 2023-07-29T13:18:41.556 INFO:teuthology.orchestra.run.smithi045.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 129 kB/s | 18 kB 00:00 2023-07-29T13:18:41.605 INFO:teuthology.orchestra.run.smithi040.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 128 kB/s | 58 kB 00:00 2023-07-29T13:18:41.615 INFO:teuthology.orchestra.run.smithi045.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2023-07-29T13:18:41.657 INFO:teuthology.orchestra.run.smithi045.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2023-07-29T13:18:41.680 INFO:teuthology.orchestra.run.smithi040.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 122 kB/s | 51 kB 00:00 2023-07-29T13:18:41.690 INFO:teuthology.orchestra.run.smithi045.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 211 kB 00:00 2023-07-29T13:18:41.716 INFO:teuthology.orchestra.run.smithi045.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 732 kB/s | 43 kB 00:00 2023-07-29T13:18:41.749 INFO:teuthology.orchestra.run.smithi045.stdout:(7/13): ceph-mgr-rook-18.1.3-11.g0484097b.el9.n 142 kB/s | 47 kB 00:00 2023-07-29T13:18:41.839 INFO:teuthology.orchestra.run.smithi040.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 620 kB/s | 548 kB 00:00 2023-07-29T13:18:41.842 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:41.843 INFO:teuthology.orchestra.run.smithi040.stdout:Total 5.9 MB/s | 6.8 MB 00:01 2023-07-29T13:18:41.843 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:18:41.854 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:18:41.854 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:18:41.949 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:18:41.950 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:18:41.975 INFO:teuthology.orchestra.run.smithi045.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 58 kB/s | 15 kB 00:00 2023-07-29T13:18:42.000 INFO:teuthology.orchestra.run.smithi045.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 104 kB/s | 32 kB 00:00 2023-07-29T13:18:42.159 INFO:teuthology.orchestra.run.smithi045.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 373 kB/s | 59 kB 00:00 2023-07-29T13:18:42.234 INFO:teuthology.orchestra.run.smithi045.stdout:(11/13): python3-google-auth-2.22.0-1.el9.noarc 406 kB/s | 197 kB 00:00 2023-07-29T13:18:42.334 INFO:teuthology.orchestra.run.smithi045.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 511 kB/s | 90 kB 00:00 2023-07-29T13:18:42.374 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:18:42.384 INFO:teuthology.orchestra.run.smithi045.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.5 MB/s | 1.0 MB 00:00 2023-07-29T13:18:42.387 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:42.387 INFO:teuthology.orchestra.run.smithi045.stdout:Total 2.0 MB/s | 2.0 MB 00:01 2023-07-29T13:18:42.387 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:18:42.404 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:18:42.404 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:18:42.506 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:18:42.506 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:18:42.562 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-07-29T13:18:42.633 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-07-29T13:18:43.170 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:18:43.198 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-07-29T13:18:43.296 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-07-29T13:18:43.357 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-07-29T13:18:43.370 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-07-29T13:18:43.403 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:18.1.3-11.g0484097b.el9.noarch 6/6 2023-07-29T13:18:43.429 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-07-29T13:18:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-07-29T13:18:43.546 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-07-29T13:18:43.596 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-07-29T13:18:43.700 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-07-29T13:18:43.836 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 7/13 2023-07-29T13:18:43.922 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-07-29T13:18:44.255 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.1.3-11.g0484097b.el9.noarch 6/6 2023-07-29T13:18:44.255 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:18.1.3-11.g0484097b.el9.noarch 1/6 2023-07-29T13:18:44.255 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-07-29T13:18:44.255 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-07-29T13:18:44.256 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-07-29T13:18:44.256 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-07-29T13:18:44.567 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-07-29T13:18:44.567 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:44.567 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:18:44.567 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:44.567 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-07-29T13:18:44.567 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2023-07-29T13:18:44.568 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-07-29T13:18:44.568 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-07-29T13:18:44.568 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-07-29T13:18:44.568 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:44.568 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:44.732 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-07-29T13:18:44.794 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-fuse 2023-07-29T13:18:44.794 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-07-29T13:18:44.844 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-07-29T13:18:44.899 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-07-29T13:18:44.930 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:18.1.3-11.g0484097b.el9.noarch 13/13 2023-07-29T13:18:45.339 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:32 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:45.474 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:45.475 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:45.475 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-29T13:18:45.475 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:45.475 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:18:45.475 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:18.1.3-11.g0484097b.el9 ceph 835 k 2023-07-29T13:18:45.476 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-29T13:18:45.476 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-07-29T13:18:45.476 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:45.476 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:18:45.476 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:45.476 INFO:teuthology.orchestra.run.smithi040.stdout:Install 2 Packages 2023-07-29T13:18:45.477 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:45.477 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 914 k 2023-07-29T13:18:45.477 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 2.7 M 2023-07-29T13:18:45.477 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:18:45.616 INFO:teuthology.orchestra.run.smithi040.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 563 kB/s | 79 kB 00:00 2023-07-29T13:18:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:18.1.3-11.g0484097b.el9.noarch 13/13 2023-07-29T13:18:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:18.1.3-11.g0484097b.el9.noarch 1/13 2023-07-29T13:18:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-07-29T13:18:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-07-29T13:18:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-07-29T13:18:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-07-29T13:18:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-07-29T13:18:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-07-29T13:18:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-07-29T13:18:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-07-29T13:18:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 10/13 2023-07-29T13:18:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-07-29T13:18:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-07-29T13:18:45.957 INFO:teuthology.orchestra.run.smithi040.stdout:(2/2): ceph-fuse-18.1.3-11.g0484097b.el9.x86_64 1.7 MB/s | 835 kB 00:00 2023-07-29T13:18:45.958 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:45.959 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.8 MB/s | 914 kB 00:00 2023-07-29T13:18:45.959 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:18:45.981 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:18:45.982 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:18:46.060 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:18:46.061 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:18:46.185 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-07-29T13:18:46.186 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:46.186 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:18:46.186 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:46.186 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-07-29T13:18:46.187 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-07-29T13:18:46.187 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-07-29T13:18:46.187 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-07-29T13:18:46.187 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-07-29T13:18:46.188 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-07-29T13:18:46.188 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-07-29T13:18:46.188 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-07-29T13:18:46.189 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-07-29T13:18:46.189 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-07-29T13:18:46.189 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2023-07-29T13:18:46.189 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-07-29T13:18:46.189 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:46.189 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:46.259 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:18:46.417 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-07-29T13:18:46.487 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-cephadm 2023-07-29T13:18:46.500 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:18.1.3-11.g0484097b.el9.x86_64 2/2 2023-07-29T13:18:47.035 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:33 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:47.171 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:47.172 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2023-07-29T13:18:47.172 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-07-29T13:18:47.172 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2023-07-29T13:18:47.173 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:18:47.173 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 127 k 2023-07-29T13:18:47.173 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-07-29T13:18:47.173 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-07-29T13:18:47.173 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-07-29T13:18:47.173 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-07-29T13:18:47.173 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-07-29T13:18:47.173 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-07-29T13:18:47.174 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:47.174 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:18:47.174 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2023-07-29T13:18:47.174 INFO:teuthology.orchestra.run.smithi045.stdout:Install 6 Packages 2023-07-29T13:18:47.174 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:47.175 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 6.8 M 2023-07-29T13:18:47.175 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 33 M 2023-07-29T13:18:47.175 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:18:47.240 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-fuse-2:18.1.3-11.g0484097b.el9.x86_64 2/2 2023-07-29T13:18:47.240 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:18.1.3-11.g0484097b.el9.x86_64 1/2 2023-07-29T13:18:47.559 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-07-29T13:18:47.559 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:47.559 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:18:47.559 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:18.1.3-11.g0484097b.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-07-29T13:18:47.560 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:47.560 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:47.892 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-volume 2023-07-29T13:18:48.087 INFO:teuthology.orchestra.run.smithi045.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.2 MB/s | 228 kB 00:00 2023-07-29T13:18:48.254 INFO:teuthology.orchestra.run.smithi045.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2023-07-29T13:18:48.354 INFO:teuthology.orchestra.run.smithi045.stdout:(3/6): ceph-mgr-cephadm-18.1.3-11.g0484097b.el9 277 kB/s | 127 kB 00:00 2023-07-29T13:18:48.444 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:35 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:48.463 INFO:teuthology.orchestra.run.smithi045.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 278 kB/s | 58 kB 00:00 2023-07-29T13:18:48.581 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:48.582 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:48.582 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-29T13:18:48.583 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:48.583 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:18:48.583 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 259 k 2023-07-29T13:18:48.584 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:48.584 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:18:48.584 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:48.584 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:18:48.585 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:48.585 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 259 k 2023-07-29T13:18:48.585 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 1.2 M 2023-07-29T13:18:48.585 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:18:48.588 INFO:teuthology.orchestra.run.smithi045.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 218 kB/s | 51 kB 00:00 2023-07-29T13:18:48.622 INFO:teuthology.orchestra.run.smithi045.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2023-07-29T13:18:48.625 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:48.625 INFO:teuthology.orchestra.run.smithi045.stdout:Total 4.7 MB/s | 6.8 MB 00:01 2023-07-29T13:18:48.626 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:18:48.635 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:18:48.635 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:18:48.726 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:18:48.727 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:18:48.947 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-volume-18.1.3-11.g0484097b.el9.noarch.rpm 711 kB/s | 259 kB 00:00 2023-07-29T13:18:48.948 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:48.948 INFO:teuthology.orchestra.run.smithi040.stdout:Total 709 kB/s | 259 kB 00:00 2023-07-29T13:18:48.948 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:18:48.956 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:18:48.957 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:18:48.988 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:18:48.989 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:18:49.022 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:18:49.205 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:18:49.212 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-07-29T13:18:49.290 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:49.291 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-07-29T13:18:49.307 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:49.307 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:49.307 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-07-29T13:18:49.307 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-07-29T13:18:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-07-29T13:18:50.028 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-07-29T13:18:50.069 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:18.1.3-11.g0484097b.el9.noarch 6/6 2023-07-29T13:18:50.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:50.378 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:50.378 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:18:50.378 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:50.379 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:50.379 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:50.719 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados-devel 2023-07-29T13:18:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.1.3-11.g0484097b.el9.noarch 6/6 2023-07-29T13:18:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:18.1.3-11.g0484097b.el9.noarch 1/6 2023-07-29T13:18:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-07-29T13:18:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-07-29T13:18:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-07-29T13:18:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-07-29T13:18:51.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-07-29T13:18:51.200 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:51.200 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:18:51.201 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:51.201 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-07-29T13:18:51.201 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2023-07-29T13:18:51.201 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-07-29T13:18:51.201 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-07-29T13:18:51.201 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-07-29T13:18:51.201 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:51.202 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:51.275 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:38 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:51.410 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:51.411 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:51.411 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:18:51.411 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:51.411 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:18:51.411 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:18.1.3-11.g0484097b.el9 ceph 126 k 2023-07-29T13:18:51.412 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:51.412 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:18:51.412 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:51.412 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:18:51.412 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:51.412 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 126 k 2023-07-29T13:18:51.413 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 456 k 2023-07-29T13:18:51.413 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:18:51.565 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-fuse 2023-07-29T13:18:51.739 INFO:teuthology.orchestra.run.smithi040.stdout:librados-devel-18.1.3-11.g0484097b.el9.x86_64.r 387 kB/s | 126 kB 00:00 2023-07-29T13:18:51.740 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:51.740 INFO:teuthology.orchestra.run.smithi040.stdout:Total 384 kB/s | 126 kB 00:00 2023-07-29T13:18:51.741 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:18:51.746 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:18:51.747 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:18:51.808 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:18:51.809 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:18:51.962 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:18:52.094 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:18:52.106 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:38 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:52.243 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:52.244 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:52.244 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-07-29T13:18:52.244 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:52.244 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:18:52.244 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:18.1.3-11.g0484097b.el9 ceph 835 k 2023-07-29T13:18:52.244 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-07-29T13:18:52.245 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-07-29T13:18:52.245 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:52.245 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:18:52.245 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:52.245 INFO:teuthology.orchestra.run.smithi045.stdout:Install 2 Packages 2023-07-29T13:18:52.245 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:52.246 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 914 k 2023-07-29T13:18:52.246 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 2.7 M 2023-07-29T13:18:52.246 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:18:52.382 INFO:teuthology.orchestra.run.smithi045.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 576 kB/s | 79 kB 00:00 2023-07-29T13:18:52.535 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:18:52.724 INFO:teuthology.orchestra.run.smithi045.stdout:(2/2): ceph-fuse-18.1.3-11.g0484097b.el9.x86_64 1.7 MB/s | 835 kB 00:00 2023-07-29T13:18:52.725 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:52.725 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.9 MB/s | 914 kB 00:00 2023-07-29T13:18:52.726 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:18:52.748 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:18:52.748 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:18:52.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:18:52.826 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:52.826 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:18:52.826 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:52.826 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:52.826 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:52.827 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:18:52.827 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:18:53.016 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:18:53.106 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install libcephfs2 2023-07-29T13:18:53.179 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-07-29T13:18:53.274 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:18.1.3-11.g0484097b.el9.x86_64 2/2 2023-07-29T13:18:53.658 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:40 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:53.742 INFO:teuthology.orchestra.run.smithi040.stdout:Package libcephfs2-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:18:53.793 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:53.794 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2023-07-29T13:18:53.794 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:53.864 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install libcephfs-devel 2023-07-29T13:18:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-fuse-2:18.1.3-11.g0484097b.el9.x86_64 2/2 2023-07-29T13:18:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:18.1.3-11.g0484097b.el9.x86_64 1/2 2023-07-29T13:18:54.332 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-07-29T13:18:54.332 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:54.333 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:18:54.333 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:18.1.3-11.g0484097b.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-07-29T13:18:54.333 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:54.333 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:54.406 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:41 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:54.541 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:54.542 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:54.542 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:18:54.543 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:54.543 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:18:54.543 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:18.1.3-11.g0484097b.el9 ceph 31 k 2023-07-29T13:18:54.543 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:54.543 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:18:54.543 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:18:54.543 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:18:54.544 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:54.544 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 31 k 2023-07-29T13:18:54.544 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 137 k 2023-07-29T13:18:54.544 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:18:54.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-volume 2023-07-29T13:18:54.791 INFO:teuthology.orchestra.run.smithi040.stdout:libcephfs-devel-18.1.3-11.g0484097b.el9.x86_64. 123 kB/s | 31 kB 00:00 2023-07-29T13:18:54.791 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:54.792 INFO:teuthology.orchestra.run.smithi040.stdout:Total 123 kB/s | 31 kB 00:00 2023-07-29T13:18:54.792 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:18:54.795 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:18:54.795 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:18:54.813 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:18:54.814 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:18:55.017 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:18:55.140 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:18:55.242 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:41 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:55.379 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:55.380 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:55.381 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-07-29T13:18:55.381 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:55.381 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:18:55.381 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:18.1.3-11.g0484097b.el9 ceph-noarch 259 k 2023-07-29T13:18:55.381 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:55.381 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:18:55.382 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:55.382 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:18:55.382 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:55.382 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 259 k 2023-07-29T13:18:55.382 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 1.2 M 2023-07-29T13:18:55.382 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:18:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:18:55.764 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-volume-18.1.3-11.g0484097b.el9.noarch.rpm 678 kB/s | 259 kB 00:00 2023-07-29T13:18:55.765 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:55.765 INFO:teuthology.orchestra.run.smithi045.stdout:Total 675 kB/s | 259 kB 00:00 2023-07-29T13:18:55.765 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:18:55.773 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:18:55.774 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:18:55.797 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:18:55.797 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:55.798 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:18:55.798 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:55.798 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:18:55.798 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:55.805 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:18:55.806 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:18:56.020 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:18:56.107 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:56.124 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:56.124 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:18:56.125 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-07-29T13:18:56.125 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:56.131 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados2 2023-07-29T13:18:56.681 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:43 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:56.764 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:18:56.817 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:56.818 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2023-07-29T13:18:56.818 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:56.884 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librbd1 2023-07-29T13:18:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:18.1.3-11.g0484097b.el9.noarch 1/1 2023-07-29T13:18:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:57.273 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:18:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:18.1.3-11.g0484097b.el9.noarch 2023-07-29T13:18:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:57.274 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:57.432 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:44 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:57.515 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:18:57.550 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados-devel 2023-07-29T13:18:57.566 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:57.567 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2023-07-29T13:18:57.567 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:57.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rados 2023-07-29T13:18:58.100 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:44 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:18:58.182 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:45 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:58.241 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:18:58.242 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:58.242 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:18:58.243 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:58.243 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:18:58.243 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:18.1.3-11.g0484097b.el9 ceph 126 k 2023-07-29T13:18:58.243 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:58.243 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:18:58.243 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:18:58.243 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:18:58.244 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:58.244 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 126 k 2023-07-29T13:18:58.244 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 456 k 2023-07-29T13:18:58.244 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:18:58.265 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rados-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:18:58.317 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:58.318 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2023-07-29T13:18:58.318 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:58.380 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rgw 2023-07-29T13:18:58.585 INFO:teuthology.orchestra.run.smithi045.stdout:librados-devel-18.1.3-11.g0484097b.el9.x86_64.r 370 kB/s | 126 kB 00:00 2023-07-29T13:18:58.586 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:18:58.586 INFO:teuthology.orchestra.run.smithi045.stdout:Total 368 kB/s | 126 kB 00:00 2023-07-29T13:18:58.586 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:18:58.592 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:18:58.593 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:18:58.656 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:18:58.656 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:18:58.803 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:18:58.909 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:18:58.930 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:45 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:59.014 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rgw-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:18:59.066 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:59.067 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2023-07-29T13:18:59.067 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:59.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-cephfs 2023-07-29T13:18:59.358 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:18:59.648 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:18:59.648 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:59.648 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:18:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:18:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:18:59.649 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:18:59.676 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:46 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:18:59.759 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-cephfs-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:18:59.812 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:18:59.813 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2023-07-29T13:18:59.813 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:18:59.883 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rbd 2023-07-29T13:18:59.904 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs2 2023-07-29T13:19:00.432 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:47 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:19:00.451 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:46 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rbd-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:19:00.535 INFO:teuthology.orchestra.run.smithi045.stdout:Package libcephfs2-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:19:00.567 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:19:00.568 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2023-07-29T13:19:00.568 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:19:00.586 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:00.587 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-07-29T13:19:00.587 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:00.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-fuse 2023-07-29T13:19:00.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs-devel 2023-07-29T13:19:01.205 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:48 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:19:01.239 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:47 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:01.343 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:19:01.343 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:01.344 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2023-07-29T13:19:01.344 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:01.344 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:19:01.344 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:18.1.3-11.g0484097b.el9 ceph 86 k 2023-07-29T13:19:01.344 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:01.344 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:19:01.345 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:01.345 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:19:01.345 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:01.345 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 86 k 2023-07-29T13:19:01.345 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 231 k 2023-07-29T13:19:01.345 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:19:01.374 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:01.375 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:01.376 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:19:01.376 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:01.376 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:19:01.376 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:18.1.3-11.g0484097b.el9 ceph 31 k 2023-07-29T13:19:01.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:01.376 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:19:01.377 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:01.377 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:19:01.377 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:01.377 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 31 k 2023-07-29T13:19:01.377 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 137 k 2023-07-29T13:19:01.377 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:19:01.613 INFO:teuthology.orchestra.run.smithi045.stdout:libcephfs-devel-18.1.3-11.g0484097b.el9.x86_64. 129 kB/s | 31 kB 00:00 2023-07-29T13:19:01.614 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:19:01.615 INFO:teuthology.orchestra.run.smithi045.stdout:Total 128 kB/s | 31 kB 00:00 2023-07-29T13:19:01.615 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:19:01.618 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:19:01.618 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:19:01.640 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:19:01.641 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:19:01.650 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-fuse-18.1.3-11.g0484097b.el9.x86_64.rpm 281 kB/s | 86 kB 00:00 2023-07-29T13:19:01.651 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:19:01.651 INFO:teuthology.orchestra.run.smithi040.stdout:Total 280 kB/s | 86 kB 00:00 2023-07-29T13:19:01.651 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:19:01.659 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:19:01.659 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:19:01.721 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:19:01.721 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:19:01.728 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:19:01.863 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:19:01.984 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:02.383 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-fuse-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:02.524 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:02.524 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:02.525 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:19:02.525 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:19:02.525 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:02.525 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:02.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados2 2023-07-29T13:19:02.810 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:02.810 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:02.810 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:19:02.810 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:19:02.810 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:02.811 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:19:03.063 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-mirror 2023-07-29T13:19:03.288 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:49 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:03.371 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:19:03.424 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:03.425 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-07-29T13:19:03.425 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:03.502 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librbd1 2023-07-29T13:19:03.612 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:50 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:19:03.748 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:19:03.748 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:03.749 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2023-07-29T13:19:03.749 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:03.749 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:19:03.750 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:18.1.3-11.g0484097b.el9 ceph 3.0 M 2023-07-29T13:19:03.750 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:03.750 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:19:03.750 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:03.751 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:19:03.751 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:03.752 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 3.0 M 2023-07-29T13:19:03.752 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 12 M 2023-07-29T13:19:03.752 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:19:04.049 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:50 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:19:04.185 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:04.186 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-07-29T13:19:04.187 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:04.258 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rados 2023-07-29T13:19:04.458 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-mirror-18.1.3-11.g0484097b.el9.x86_64.rpm 4.2 MB/s | 3.0 MB 00:00 2023-07-29T13:19:04.459 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:19:04.459 INFO:teuthology.orchestra.run.smithi040.stdout:Total 4.2 MB/s | 3.0 MB 00:00 2023-07-29T13:19:04.459 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:19:04.470 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:19:04.470 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:19:04.556 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:19:04.557 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:19:04.798 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:50 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:04.881 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rados-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:19:04.934 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:04.935 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-07-29T13:19:04.936 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:04.998 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:19:05.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rgw 2023-07-29T13:19:05.086 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:05.112 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:05.112 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:19:05.112 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-07-29T13:19:05.113 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-07-29T13:19:05.113 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-07-29T13:19:05.113 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:05.560 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:51 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:05.643 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rgw-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:19:05.694 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:05.695 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-07-29T13:19:05.696 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:05.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-cephfs 2023-07-29T13:19:06.311 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:52 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:06.313 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:06.313 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:06.313 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:19:06.314 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:19:06.314 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:06.314 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:19:06.395 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-cephfs-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:19:06.447 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:06.448 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-07-29T13:19:06.449 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:06.525 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rbd 2023-07-29T13:19:06.584 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-nbd 2023-07-29T13:19:07.074 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:53 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:07.133 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:54 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:19:07.158 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rbd-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:19:07.211 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:07.212 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-07-29T13:19:07.212 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:07.270 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:19:07.271 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2023-07-29T13:19:07.271 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:07.271 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:19:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:18.1.3-11.g0484097b.el9 ceph 170 k 2023-07-29T13:19:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:07.272 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:19:07.272 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:07.272 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:19:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:07.273 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 170 k 2023-07-29T13:19:07.273 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 490 k 2023-07-29T13:19:07.273 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:19:07.276 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-fuse 2023-07-29T13:19:07.764 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-nbd-18.1.3-11.g0484097b.el9.x86_64.rpm 344 kB/s | 170 kB 00:00 2023-07-29T13:19:07.765 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:19:07.765 INFO:teuthology.orchestra.run.smithi040.stdout:Total 343 kB/s | 170 kB 00:00 2023-07-29T13:19:07.766 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:19:07.774 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:19:07.774 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:19:07.823 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:53 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:07.836 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:19:07.837 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:19:07.960 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:07.961 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2023-07-29T13:19:07.961 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:07.961 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:19:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:18.1.3-11.g0484097b.el9 ceph 86 k 2023-07-29T13:19:07.962 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:07.962 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:19:07.962 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:07.962 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:19:07.962 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:07.962 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 86 k 2023-07-29T13:19:07.962 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 231 k 2023-07-29T13:19:07.963 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:19:07.995 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:19:08.088 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:08.291 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-fuse-18.1.3-11.g0484097b.el9.x86_64.rpm 261 kB/s | 86 kB 00:00 2023-07-29T13:19:08.292 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:19:08.292 INFO:teuthology.orchestra.run.smithi045.stdout:Total 260 kB/s | 86 kB 00:00 2023-07-29T13:19:08.293 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:19:08.300 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:19:08.301 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:19:08.363 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:19:08.364 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:19:08.512 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:19:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-nbd-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:08.624 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:08.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:08.825 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:08.825 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:19:08.825 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:19:08.826 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:08.826 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:19:08.971 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-fuse-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:09.103 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-base-debuginfo 2023-07-29T13:19:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:09.272 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:19:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:19:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:09.272 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:09.560 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-mirror 2023-07-29T13:19:09.653 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:56 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:19:09.789 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:19:09.790 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:19:09.790 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:09.791 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:19:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 74 M 2023-07-29T13:19:09.791 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-29T13:19:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 20 M 2023-07-29T13:19:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:09.791 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:19:09.791 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:09.792 INFO:teuthology.orchestra.run.smithi040.stdout:Install 2 Packages 2023-07-29T13:19:09.792 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:09.792 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 94 M 2023-07-29T13:19:09.793 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 512 M 2023-07-29T13:19:09.793 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:19:10.110 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:56 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:10.245 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:10.246 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:10.246 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2023-07-29T13:19:10.246 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:10.247 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:19:10.247 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:18.1.3-11.g0484097b.el9 ceph 3.0 M 2023-07-29T13:19:10.247 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:10.247 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:19:10.247 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:10.247 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:19:10.247 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:10.248 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 3.0 M 2023-07-29T13:19:10.248 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 12 M 2023-07-29T13:19:10.248 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:19:11.383 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-mirror-18.1.3-11.g0484097b.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2023-07-29T13:19:11.384 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:19:11.385 INFO:teuthology.orchestra.run.smithi045.stdout:Total 2.6 MB/s | 3.0 MB 00:01 2023-07-29T13:19:11.385 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:19:11.397 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:19:11.397 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:19:11.487 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:19:11.487 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:19:11.941 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:19:12.001 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:12.027 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:12.027 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-07-29T13:19:12.027 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-07-29T13:19:12.028 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-07-29T13:19:12.028 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-07-29T13:19:12.028 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:12.577 INFO:teuthology.orchestra.run.smithi040.stdout:(1/2): ceph-debuginfo-18.1.3-11.g0484097b.el9.x 7.4 MB/s | 20 MB 00:02 2023-07-29T13:19:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:13.320 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:19:13.320 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:19:13.321 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:13.321 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:13.597 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-nbd 2023-07-29T13:19:14.145 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:00 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:14.281 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:14.282 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2023-07-29T13:19:14.283 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:14.283 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:19:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:18.1.3-11.g0484097b.el9 ceph 170 k 2023-07-29T13:19:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:14.284 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:19:14.284 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:14.284 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:19:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:14.285 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 170 k 2023-07-29T13:19:14.285 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 490 k 2023-07-29T13:19:14.285 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:19:14.610 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-nbd-18.1.3-11.g0484097b.el9.x86_64.rpm 518 kB/s | 170 kB 00:00 2023-07-29T13:19:14.611 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:19:14.611 INFO:teuthology.orchestra.run.smithi045.stdout:Total 515 kB/s | 170 kB 00:00 2023-07-29T13:19:14.612 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:19:14.621 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:19:14.621 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:19:14.683 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:19:14.684 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:19:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:19:14.943 INFO:teuthology.orchestra.run.smithi040.stdout:(2/2): ceph-base-debuginfo-18.1.3-11.g0484097b. 14 MB/s | 74 MB 00:05 2023-07-29T13:19:14.945 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:19:14.946 INFO:teuthology.orchestra.run.smithi040.stdout:Total 18 MB/s | 94 MB 00:05 2023-07-29T13:19:14.946 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:19:14.951 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:19:14.951 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:19:14.968 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:15.391 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-nbd-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:15.583 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:19:15.584 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:19:15.709 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:19:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:15.710 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:19:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:19:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:15.710 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:16.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-base-debuginfo 2023-07-29T13:19:16.561 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:02 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:16.698 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:16.698 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:16.699 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:19:16.699 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:16.699 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:19:16.699 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 74 M 2023-07-29T13:19:16.699 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-07-29T13:19:16.699 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 20 M 2023-07-29T13:19:16.699 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:16.700 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:19:16.700 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:16.700 INFO:teuthology.orchestra.run.smithi045.stdout:Install 2 Packages 2023-07-29T13:19:16.700 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:16.701 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 94 M 2023-07-29T13:19:16.701 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 512 M 2023-07-29T13:19:16.701 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:19:18.231 INFO:teuthology.orchestra.run.smithi045.stdout:(1/2): ceph-debuginfo-18.1.3-11.g0484097b.el9.x 13 MB/s | 20 MB 00:01 2023-07-29T13:19:18.650 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:19:22.623 INFO:teuthology.orchestra.run.smithi045.stdout:(2/2): ceph-base-debuginfo-18.1.3-11.g0484097b. 12 MB/s | 74 MB 00:05 2023-07-29T13:19:22.625 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:19:22.625 INFO:teuthology.orchestra.run.smithi045.stdout:Total 16 MB/s | 94 MB 00:05 2023-07-29T13:19:22.625 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:19:22.630 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:19:22.630 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:19:23.268 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:19:23.269 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:19:26.341 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:19:26.757 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/2 2023-07-29T13:19:26.828 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2/2 2023-07-29T13:19:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2/2 2023-07-29T13:19:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/2 2023-07-29T13:19:30.566 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2/2 2023-07-29T13:19:30.566 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:30.567 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:19:30.567 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:19:30.567 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:19:30.567 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:30.567 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:19:30.819 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-common-debuginfo 2023-07-29T13:19:31.363 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:18 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:19:31.498 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:19:31.499 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:31.499 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:19:31.499 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:31.499 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:19:31.499 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 403 M 2023-07-29T13:19:31.500 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:31.500 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:19:31.500 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:19:31.500 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:19:31.500 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:19:31.500 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 403 M 2023-07-29T13:19:31.501 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 1.7 G 2023-07-29T13:19:31.501 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:19:34.461 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/2 2023-07-29T13:19:34.550 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2/2 2023-07-29T13:19:38.462 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2/2 2023-07-29T13:19:38.462 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/2 2023-07-29T13:19:38.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2/2 2023-07-29T13:19:38.741 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:38.741 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:19:38.742 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:19:38.742 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:19:38.742 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:38.742 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:19:39.008 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-common-debuginfo 2023-07-29T13:19:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:25 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:19:39.691 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:19:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:39.692 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:19:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:39.692 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:19:39.692 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 403 M 2023-07-29T13:19:39.692 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:19:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:19:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:19:39.693 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:19:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 403 M 2023-07-29T13:19:39.693 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 1.7 G 2023-07-29T13:19:39.694 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:19:51.439 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-common-debuginfo-18.1.3-11.g0484097b.el9.x 20 MB/s | 403 MB 00:19 2023-07-29T13:19:51.440 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:19:51.440 INFO:teuthology.orchestra.run.smithi040.stdout:Total 20 MB/s | 403 MB 00:19 2023-07-29T13:19:51.441 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:19:51.445 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:19:51.445 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:19:53.973 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:19:53.973 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:20:09.266 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-common-debuginfo-18.1.3-11.g0484097b.el9.x 14 MB/s | 403 MB 00:29 2023-07-29T13:20:09.267 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:20:09.268 INFO:teuthology.orchestra.run.smithi045.stdout:Total 14 MB/s | 403 MB 00:29 2023-07-29T13:20:09.268 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:20:09.273 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:20:09.273 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:20:11.801 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:20:11.801 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:20:37.922 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:20:38.035 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:20:47.164 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:20:47.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:20:47.490 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:20:47.490 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:20:47.490 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:20:47.490 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:20:47.491 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:20:47.791 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2023-07-29T13:20:48.337 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:03:35 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:20:48.471 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:20:48.472 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2023-07-29T13:20:48.472 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:20:48.473 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2023-07-29T13:20:48.473 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:20:48.473 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 1.4 M 2023-07-29T13:20:48.473 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:20:48.473 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:20:48.473 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2023-07-29T13:20:48.473 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:20:48.474 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:20:48.474 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 1.4 M 2023-07-29T13:20:48.474 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 6.2 M 2023-07-29T13:20:48.474 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:20:48.973 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-immutable-object-cache-debuginfo-18.1.3-11 2.8 MB/s | 1.4 MB 00:00 2023-07-29T13:20:48.974 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:20:48.974 INFO:teuthology.orchestra.run.smithi040.stdout:Total 2.8 MB/s | 1.4 MB 00:00 2023-07-29T13:20:48.974 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:20:48.977 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:20:48.978 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:20:49.023 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:20:49.023 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:20:49.321 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:20:49.439 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:18.1.3-11.g0 1/1 2023-07-29T13:20:49.900 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:18.1.3-11.g0 1/1 2023-07-29T13:20:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:18.1.3-11.g0 1/1 2023-07-29T13:20:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:20:50.165 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:20:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:20:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:20:50.166 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:20:50.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw-debuginfo 2023-07-29T13:20:50.917 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:03:37 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:20:51.053 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:20:51.054 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:20:51.054 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:20:51.054 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:20:51.054 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:20:51.054 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 162 M 2023-07-29T13:20:51.055 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:20:51.055 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:20:51.055 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:20:51.055 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:20:51.055 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:20:51.055 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 162 M 2023-07-29T13:20:51.055 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 675 M 2023-07-29T13:20:51.055 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:20:55.649 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:20:55.746 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:20:58.601 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-radosgw-debuginfo-18.1.3-11.g0484097b.el9. 21 MB/s | 162 MB 00:07 2023-07-29T13:20:58.601 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:20:58.602 INFO:teuthology.orchestra.run.smithi040.stdout:Total 21 MB/s | 162 MB 00:07 2023-07-29T13:20:58.602 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:20:58.606 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:20:58.607 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:20:59.640 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:20:59.641 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:21:03.783 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:21:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:21:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:04.030 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:21:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:21:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:04.031 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:21:04.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2023-07-29T13:21:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:03:50 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:21:04.972 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:21:04.972 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2023-07-29T13:21:04.973 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:21:04.973 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2023-07-29T13:21:04.973 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:21:04.973 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 1.4 M 2023-07-29T13:21:04.973 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:04.973 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:21:04.973 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2023-07-29T13:21:04.974 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:21:04.974 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:04.974 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 1.4 M 2023-07-29T13:21:04.974 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 6.2 M 2023-07-29T13:21:04.974 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:21:05.516 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-immutable-object-cache-debuginfo-18.1.3-11 2.6 MB/s | 1.4 MB 00:00 2023-07-29T13:21:05.517 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:21:05.517 INFO:teuthology.orchestra.run.smithi045.stdout:Total 2.6 MB/s | 1.4 MB 00:00 2023-07-29T13:21:05.518 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:21:05.520 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:21:05.520 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:21:05.565 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:21:05.566 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:21:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:21:05.931 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:18.1.3-11.g0 1/1 2023-07-29T13:21:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:18.1.3-11.g0 1/1 2023-07-29T13:21:06.712 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:18.1.3-11.g0 1/1 2023-07-29T13:21:06.712 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:06.712 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:21:06.712 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:21:06.713 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:06.713 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:21:07.075 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw-debuginfo 2023-07-29T13:21:07.625 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:03:53 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:21:07.766 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:21:07.766 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:21:07.766 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:21:07.766 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:21:07.767 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:21:07.767 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 162 M 2023-07-29T13:21:07.767 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:07.767 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:21:07.767 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:21:07.767 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:21:07.767 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:07.768 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 162 M 2023-07-29T13:21:07.768 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 675 M 2023-07-29T13:21:07.768 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:21:17.101 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:21:17.216 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86 1/1 2023-07-29T13:21:21.515 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86 1/1 2023-07-29T13:21:21.782 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86 1/1 2023-07-29T13:21:21.782 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:21:21.782 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:21:21.783 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:21:21.783 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:21:21.783 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:21:22.005 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-test-debuginfo 2023-07-29T13:21:22.555 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:04:09 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:21:22.692 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:21:22.693 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:21:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:21:22.693 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:21:22.693 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:21:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 600 M 2023-07-29T13:21:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:21:22.694 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:21:22.694 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:21:22.694 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:21:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:21:22.694 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 600 M 2023-07-29T13:21:22.694 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 2.7 G 2023-07-29T13:21:22.694 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:21:29.967 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-radosgw-debuginfo-18.1.3-11.g0484097b.el9. 7.3 MB/s | 162 MB 00:22 2023-07-29T13:21:29.968 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:21:29.968 INFO:teuthology.orchestra.run.smithi045.stdout:Total 7.3 MB/s | 162 MB 00:22 2023-07-29T13:21:29.969 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:21:29.973 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:21:30.042 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:21:31.008 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:21:31.009 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:21:48.492 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:21:48.590 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86 1/1 2023-07-29T13:21:53.246 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86 1/1 2023-07-29T13:21:53.532 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86 1/1 2023-07-29T13:21:53.532 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:53.532 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:21:53.532 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:21:53.532 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:53.533 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:21:53.756 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-test-debuginfo 2023-07-29T13:21:54.301 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:04:40 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:21:54.437 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:21:54.438 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:21:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:21:54.438 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:21:54.439 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:21:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 600 M 2023-07-29T13:21:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:54.439 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:21:54.439 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:21:54.440 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:21:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:21:54.440 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 600 M 2023-07-29T13:21:54.440 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 2.7 G 2023-07-29T13:21:54.440 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:21:58.825 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-test-debuginfo-18.1.3-11.g0484097b.el9.x86 17 MB/s | 600 MB 00:36 2023-07-29T13:21:58.826 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:21:58.826 INFO:teuthology.orchestra.run.smithi040.stdout:Total 17 MB/s | 600 MB 00:36 2023-07-29T13:21:58.827 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:21:58.833 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:21:58.833 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:22:02.575 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:22:02.575 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:22:20.890 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-test-debuginfo-18.1.3-11.g0484097b.el9.x86 23 MB/s | 600 MB 00:26 2023-07-29T13:22:20.891 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:22:20.892 INFO:teuthology.orchestra.run.smithi045.stdout:Total 23 MB/s | 600 MB 00:26 2023-07-29T13:22:20.892 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:22:20.898 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:22:20.898 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:22:24.644 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:22:24.644 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:23:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:23:09.762 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-test-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:17.634 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:17.634 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:17.634 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:23:17.634 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:23:17.635 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:17.635 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:23:17.907 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-base-debuginfo 2023-07-29T13:23:18.461 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:06:05 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:23:18.544 INFO:teuthology.orchestra.run.smithi040.stdout:Package ceph-base-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:23:18.595 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:23:18.596 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2023-07-29T13:23:18.596 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:23:18.661 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-debuginfo 2023-07-29T13:23:19.205 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:06:06 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:23:19.340 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:23:19.341 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:19.341 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:23:19.341 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:19.341 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:23:19.342 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 19 M 2023-07-29T13:23:19.342 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:19.342 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:23:19.342 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:19.342 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:23:19.342 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:19.342 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 19 M 2023-07-29T13:23:19.343 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 101 M 2023-07-29T13:23:19.343 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:23:20.564 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-mgr-debuginfo-18.1.3-11.g0484097b.el9.x86_ 16 MB/s | 19 MB 00:01 2023-07-29T13:23:20.565 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:23:20.565 INFO:teuthology.orchestra.run.smithi040.stdout:Total 16 MB/s | 19 MB 00:01 2023-07-29T13:23:20.565 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:23:20.569 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:23:20.569 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:23:20.724 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:23:20.724 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:23:23.035 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:23:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:24.114 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:24.366 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:24.366 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:24.366 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:23:24.367 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:23:24.367 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:24.367 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:23:24.573 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mds-debuginfo 2023-07-29T13:23:25.119 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:06:12 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:23:25.255 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:23:25.256 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:25.256 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:23:25.257 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:25.257 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:23:25.257 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 48 M 2023-07-29T13:23:25.257 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:25.257 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:23:25.257 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:25.258 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:23:25.258 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:25.258 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 48 M 2023-07-29T13:23:25.258 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 203 M 2023-07-29T13:23:25.258 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:23:27.809 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-mds-debuginfo-18.1.3-11.g0484097b.el9.x86_ 19 MB/s | 48 MB 00:02 2023-07-29T13:23:27.810 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:23:27.810 INFO:teuthology.orchestra.run.smithi040.stdout:Total 19 MB/s | 48 MB 00:02 2023-07-29T13:23:27.811 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:23:27.815 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:23:27.815 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:23:28.150 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:23:28.151 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:23:31.299 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:23:31.425 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:33.443 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:23:33.542 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:35.108 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:35.347 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:35.348 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:35.348 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:23:35.348 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:23:35.348 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:35.348 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:23:35.564 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mon-debuginfo 2023-07-29T13:23:36.113 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:06:23 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:23:36.249 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:23:36.250 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:23:36.250 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:36.250 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:23:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 113 M 2023-07-29T13:23:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:36.251 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:23:36.251 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:36.251 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:23:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:36.252 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 113 M 2023-07-29T13:23:36.252 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 525 M 2023-07-29T13:23:36.252 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:23:40.152 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-test-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:40.471 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:40.472 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:40.472 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:23:40.472 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:23:40.472 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:40.472 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:23:40.756 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-base-debuginfo 2023-07-29T13:23:41.302 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:06:27 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:23:41.384 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-base-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 is already installed. 2023-07-29T13:23:41.435 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:23:41.436 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-07-29T13:23:41.437 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:23:41.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-debuginfo 2023-07-29T13:23:41.528 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-mon-debuginfo-18.1.3-11.g0484097b.el9.x86_ 21 MB/s | 113 MB 00:05 2023-07-29T13:23:41.529 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:23:41.530 INFO:teuthology.orchestra.run.smithi040.stdout:Total 21 MB/s | 113 MB 00:05 2023-07-29T13:23:41.530 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:23:41.534 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:23:41.534 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:23:42.058 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:06:28 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:23:42.194 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:23:42.195 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:23:42.195 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:23:42.195 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:23:42.195 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:23:42.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 19 M 2023-07-29T13:23:42.195 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:42.196 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:23:42.196 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:23:42.196 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:23:42.196 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:42.196 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 19 M 2023-07-29T13:23:42.196 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 101 M 2023-07-29T13:23:42.197 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:23:42.265 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:23:42.266 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:23:43.436 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mgr-debuginfo-18.1.3-11.g0484097b.el9.x86_ 15 MB/s | 19 MB 00:01 2023-07-29T13:23:43.437 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:23:43.438 INFO:teuthology.orchestra.run.smithi045.stdout:Total 15 MB/s | 19 MB 00:01 2023-07-29T13:23:43.438 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:23:43.442 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:23:43.442 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:23:43.598 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:23:43.598 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:23:45.921 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:23:46.027 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:47.021 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:47.271 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:47.272 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:47.272 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:23:47.272 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:23:47.272 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:47.272 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:23:47.459 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mds-debuginfo 2023-07-29T13:23:48.012 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:06:34 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:23:48.150 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:23:48.150 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:23:48.151 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:23:48.151 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:23:48.151 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:23:48.151 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 48 M 2023-07-29T13:23:48.151 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:48.151 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:23:48.152 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:23:48.152 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:23:48.152 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:48.152 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 48 M 2023-07-29T13:23:48.152 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 203 M 2023-07-29T13:23:48.152 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:23:50.739 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mds-debuginfo-18.1.3-11.g0484097b.el9.x86_ 19 MB/s | 48 MB 00:02 2023-07-29T13:23:50.740 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:23:50.741 INFO:teuthology.orchestra.run.smithi045.stdout:Total 19 MB/s | 48 MB 00:02 2023-07-29T13:23:50.741 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:23:50.745 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:23:50.745 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:23:51.081 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:23:51.082 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:23:54.845 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:23:54.934 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:56.349 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:23:56.446 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:58.058 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:58.311 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:58.311 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:58.311 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:23:58.311 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:23:58.312 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:58.312 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:23:58.401 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:58.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mon-debuginfo 2023-07-29T13:23:58.643 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:23:58.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:58.643 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:23:58.644 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:23:58.644 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:58.644 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:23:58.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-osd-debuginfo 2023-07-29T13:23:59.075 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:06:45 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:23:59.211 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:23:59.212 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:23:59.212 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:23:59.212 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:23:59.212 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:23:59.213 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 113 M 2023-07-29T13:23:59.213 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:59.213 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:23:59.213 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:23:59.213 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:23:59.213 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:23:59.214 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 113 M 2023-07-29T13:23:59.214 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 525 M 2023-07-29T13:23:59.214 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:23:59.395 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:06:46 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:23:59.533 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:23:59.534 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:59.534 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:23:59.534 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:59.534 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:23:59.535 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 312 M 2023-07-29T13:23:59.535 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:59.535 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:23:59.535 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:23:59.535 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:23:59.535 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:23:59.535 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 312 M 2023-07-29T13:23:59.536 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 1.3 G 2023-07-29T13:23:59.536 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:24:04.625 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon-debuginfo-18.1.3-11.g0484097b.el9.x86_ 21 MB/s | 113 MB 00:05 2023-07-29T13:24:04.626 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:24:04.626 INFO:teuthology.orchestra.run.smithi045.stdout:Total 21 MB/s | 113 MB 00:05 2023-07-29T13:24:04.627 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:24:04.631 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:24:04.631 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:24:05.362 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:24:05.363 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:24:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:24:18.007 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:24:21.417 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:24:21.693 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:24:21.693 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:24:21.694 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:24:21.694 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:24:21.694 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:24:21.694 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:24:21.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-osd-debuginfo 2023-07-29T13:24:22.448 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:07:08 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:24:22.587 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:24:22.588 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:24:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:24:22.588 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:24:22.588 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:24:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 312 M 2023-07-29T13:24:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:24:22.589 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:24:22.589 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:24:22.589 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:24:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:24:22.589 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 312 M 2023-07-29T13:24:22.589 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 1.3 G 2023-07-29T13:24:22.589 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:24:40.501 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-osd-debuginfo-18.1.3-11.g0484097b.el9.x86_ 17 MB/s | 312 MB 00:17 2023-07-29T13:24:40.501 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:24:40.502 INFO:teuthology.orchestra.run.smithi045.stdout:Total 17 MB/s | 312 MB 00:17 2023-07-29T13:24:40.502 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:24:40.507 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:24:40.507 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:24:41.818 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-osd-debuginfo-18.1.3-11.g0484097b.el9.x86_ 7.4 MB/s | 312 MB 00:42 2023-07-29T13:24:41.819 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:24:41.819 INFO:teuthology.orchestra.run.smithi040.stdout:Total 7.4 MB/s | 312 MB 00:42 2023-07-29T13:24:41.820 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:24:41.824 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:24:41.824 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:24:42.472 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:24:42.473 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:24:43.786 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:24:43.786 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:25:16.443 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:25:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:17.607 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:25:17.824 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:24.681 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:24.940 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:24.941 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:24.941 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:25:24.941 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:24.941 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:24.941 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:25:25.160 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-fuse-debuginfo 2023-07-29T13:25:25.170 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:25.442 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:25.442 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:25.442 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:25:25.442 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:25.442 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:25.443 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:25:25.678 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-fuse-debuginfo 2023-07-29T13:25:25.719 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:08:11 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:25:25.853 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:25:25.854 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:25.854 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:25:25.854 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:25.854 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:25:25.854 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 5.7 M 2023-07-29T13:25:25.855 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:25.855 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:25:25.855 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:25.855 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:25:25.855 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:25.855 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 5.7 M 2023-07-29T13:25:25.855 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 27 M 2023-07-29T13:25:25.855 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:25:26.228 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:08:13 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:25:26.364 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:25:26.365 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:26.365 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:25:26.365 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:26.365 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:25:26.366 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 5.7 M 2023-07-29T13:25:26.366 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:26.366 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:25:26.366 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:26.366 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:25:26.366 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:26.366 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 5.7 M 2023-07-29T13:25:26.367 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 27 M 2023-07-29T13:25:26.367 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:25:26.619 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-fuse-debuginfo-18.1.3-11.g0484097b.el9.x86 7.5 MB/s | 5.7 MB 00:00 2023-07-29T13:25:26.620 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:25:26.620 INFO:teuthology.orchestra.run.smithi045.stdout:Total 7.4 MB/s | 5.7 MB 00:00 2023-07-29T13:25:26.620 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:25:26.624 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:25:26.624 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:25:26.695 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:25:26.696 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:25:27.087 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-fuse-debuginfo-18.1.3-11.g0484097b.el9.x86 7.9 MB/s | 5.7 MB 00:00 2023-07-29T13:25:27.088 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:25:27.088 INFO:teuthology.orchestra.run.smithi040.stdout:Total 7.9 MB/s | 5.7 MB 00:00 2023-07-29T13:25:27.088 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:25:27.092 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:25:27.092 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:25:27.164 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:25:27.164 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:25:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:25:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:27.926 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:25:28.054 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:28.134 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:28.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:28.506 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:28.506 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:25:28.506 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:28.507 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:28.507 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:25:28.661 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:28.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados-devel-debuginfo 2023-07-29T13:25:28.916 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:28.917 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:28.917 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:25:28.917 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:28.917 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:28.917 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:25:29.111 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados-devel-debuginfo 2023-07-29T13:25:29.232 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:08:15 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:25:29.368 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:25:29.369 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:29.369 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:25:29.369 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:29.370 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:25:29.370 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 108 k 2023-07-29T13:25:29.370 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:29.370 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:25:29.370 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:29.370 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:25:29.370 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:29.371 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 108 k 2023-07-29T13:25:29.371 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 440 k 2023-07-29T13:25:29.371 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:25:29.660 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:08:16 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:25:29.796 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:25:29.797 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:29.797 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:25:29.797 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:29.797 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:25:29.798 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 108 k 2023-07-29T13:25:29.798 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:29.798 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:25:29.798 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:29.798 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:25:29.798 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:29.798 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 108 k 2023-07-29T13:25:29.798 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 440 k 2023-07-29T13:25:29.799 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:25:29.822 INFO:teuthology.orchestra.run.smithi045.stdout:librados-devel-debuginfo-18.1.3-11.g0484097b.el 239 kB/s | 108 kB 00:00 2023-07-29T13:25:29.823 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:25:29.823 INFO:teuthology.orchestra.run.smithi045.stdout:Total 238 kB/s | 108 kB 00:00 2023-07-29T13:25:29.823 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:25:29.827 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:25:29.827 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:25:29.864 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:25:29.865 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:25:29.987 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:25:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:25:30.132 INFO:teuthology.orchestra.run.smithi040.stdout:librados-devel-debuginfo-18.1.3-11.g0484097b.el 324 kB/s | 108 kB 00:00 2023-07-29T13:25:30.133 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:25:30.133 INFO:teuthology.orchestra.run.smithi040.stdout:Total 323 kB/s | 108 kB 00:00 2023-07-29T13:25:30.133 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:25:30.136 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:25:30.136 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:25:30.174 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:25:30.174 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:25:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:25:30.390 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:25:30.468 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados-devel-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:25:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados-devel-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:25:30.825 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:25:30.825 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:30.825 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:25:30.826 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:30.826 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:30.826 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:25:31.019 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs2-debuginfo 2023-07-29T13:25:31.080 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:25:31.080 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:31.080 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:25:31.080 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:31.080 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:31.081 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:25:31.284 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install libcephfs2-debuginfo 2023-07-29T13:25:31.563 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:08:17 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:25:31.699 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:25:31.700 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:31.700 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:25:31.700 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:31.701 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:25:31.701 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 4.8 M 2023-07-29T13:25:31.701 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:31.701 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:25:31.701 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:31.701 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:25:31.701 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:31.701 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 4.8 M 2023-07-29T13:25:31.702 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 23 M 2023-07-29T13:25:31.702 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:25:31.824 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:08:18 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:25:31.958 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:25:31.959 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:31.959 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:25:31.959 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:31.959 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:25:31.959 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 4.8 M 2023-07-29T13:25:31.960 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:31.960 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:25:31.960 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:31.960 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:25:31.960 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:31.960 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 4.8 M 2023-07-29T13:25:31.961 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 23 M 2023-07-29T13:25:31.961 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:25:32.334 INFO:teuthology.orchestra.run.smithi045.stdout:libcephfs2-debuginfo-18.1.3-11.g0484097b.el9.x8 7.7 MB/s | 4.8 MB 00:00 2023-07-29T13:25:32.335 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:25:32.335 INFO:teuthology.orchestra.run.smithi045.stdout:Total 7.6 MB/s | 4.8 MB 00:00 2023-07-29T13:25:32.335 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:25:32.339 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:25:32.339 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:25:32.407 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:25:32.407 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:25:32.627 INFO:teuthology.orchestra.run.smithi040.stdout:libcephfs2-debuginfo-18.1.3-11.g0484097b.el9.x8 7.3 MB/s | 4.8 MB 00:00 2023-07-29T13:25:32.628 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:25:32.628 INFO:teuthology.orchestra.run.smithi040.stdout:Total 7.2 MB/s | 4.8 MB 00:00 2023-07-29T13:25:32.629 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:25:32.632 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:25:32.633 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:25:32.698 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:25:32.698 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:25:33.056 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:25:33.175 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:25:33.351 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:25:33.478 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:25:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:25:33.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:25:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:33.948 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:25:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:33.948 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:33.948 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:25:34.032 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:25:34.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados2-debuginfo 2023-07-29T13:25:34.308 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:25:34.309 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:34.309 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:25:34.309 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:34.309 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:34.309 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:25:34.510 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados2-debuginfo 2023-07-29T13:25:34.707 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:08:20 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:25:34.841 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:25:34.842 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:34.842 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:25:34.842 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:34.842 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:25:34.843 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 44 M 2023-07-29T13:25:34.843 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:34.843 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:25:34.843 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:34.843 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:25:34.843 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:34.843 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 44 M 2023-07-29T13:25:34.844 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 238 M 2023-07-29T13:25:34.844 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:25:35.063 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:08:22 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:25:35.199 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:25:35.200 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:35.200 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:25:35.200 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:35.200 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:25:35.200 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 44 M 2023-07-29T13:25:35.200 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:35.201 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:25:35.201 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:35.201 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:25:35.201 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:35.201 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 44 M 2023-07-29T13:25:35.201 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 238 M 2023-07-29T13:25:35.202 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:25:37.237 INFO:teuthology.orchestra.run.smithi045.stdout:librados2-debuginfo-18.1.3-11.g0484097b.el9.x86 18 MB/s | 44 MB 00:02 2023-07-29T13:25:37.238 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:25:37.238 INFO:teuthology.orchestra.run.smithi045.stdout:Total 18 MB/s | 44 MB 00:02 2023-07-29T13:25:37.239 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:25:37.243 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:25:37.243 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:25:37.560 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:25:37.560 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:25:39.380 INFO:teuthology.orchestra.run.smithi040.stdout:librados2-debuginfo-18.1.3-11.g0484097b.el9.x86 11 MB/s | 44 MB 00:04 2023-07-29T13:25:39.381 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:25:39.381 INFO:teuthology.orchestra.run.smithi040.stdout:Total 11 MB/s | 44 MB 00:04 2023-07-29T13:25:39.382 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:25:39.386 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:25:39.386 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:25:39.702 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:25:39.703 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:25:42.776 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:25:42.879 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:44.678 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:44.926 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:25:44.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:44.954 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:44.954 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:25:44.954 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:44.954 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:44.954 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:25:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:45.186 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librbd1-debuginfo 2023-07-29T13:25:45.732 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:08:31 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:25:45.866 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:25:45.867 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:25:45.868 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:45.868 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:25:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 51 M 2023-07-29T13:25:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:45.869 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:25:45.869 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:45.869 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:25:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:45.871 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 51 M 2023-07-29T13:25:45.871 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 292 M 2023-07-29T13:25:45.871 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:25:46.920 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:47.161 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:47.162 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:47.162 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:25:47.162 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:47.162 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:47.162 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:25:47.394 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librbd1-debuginfo 2023-07-29T13:25:47.949 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:08:34 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:25:48.083 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:25:48.084 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:48.084 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:25:48.084 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:48.085 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:25:48.085 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 51 M 2023-07-29T13:25:48.085 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:48.085 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:25:48.085 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:25:48.085 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:25:48.085 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:48.086 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 51 M 2023-07-29T13:25:48.086 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 292 M 2023-07-29T13:25:48.086 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:25:48.498 INFO:teuthology.orchestra.run.smithi045.stdout:librbd1-debuginfo-18.1.3-11.g0484097b.el9.x86_6 19 MB/s | 51 MB 00:02 2023-07-29T13:25:48.499 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:25:48.500 INFO:teuthology.orchestra.run.smithi045.stdout:Total 19 MB/s | 51 MB 00:02 2023-07-29T13:25:48.500 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:25:48.504 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:25:48.504 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:25:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:25:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:25:50.717 INFO:teuthology.orchestra.run.smithi040.stdout:librbd1-debuginfo-18.1.3-11.g0484097b.el9.x86_6 19 MB/s | 51 MB 00:02 2023-07-29T13:25:50.718 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:25:50.718 INFO:teuthology.orchestra.run.smithi040.stdout:Total 19 MB/s | 51 MB 00:02 2023-07-29T13:25:50.719 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:25:50.723 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:25:50.723 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:25:51.079 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:25:51.080 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:25:54.995 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:25:55.099 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librbd1-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:57.113 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:57.228 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:25:57.332 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librbd1-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:57.379 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:57.379 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:57.379 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:25:57.379 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:57.380 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:57.380 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:25:57.620 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-cephfs-debuginfo 2023-07-29T13:25:58.158 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:08:44 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:25:58.295 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:25:58.296 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:58.296 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:25:58.296 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:58.296 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:25:58.296 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 593 k 2023-07-29T13:25:58.297 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:58.297 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:25:58.297 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:25:58.297 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:25:58.297 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:25:58.297 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 593 k 2023-07-29T13:25:58.297 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 2.0 M 2023-07-29T13:25:58.298 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:25:59.067 INFO:teuthology.orchestra.run.smithi045.stdout:python3-cephfs-debuginfo-18.1.3-11.g0484097b.el 769 kB/s | 593 kB 00:00 2023-07-29T13:25:59.068 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:25:59.068 INFO:teuthology.orchestra.run.smithi045.stdout:Total 767 kB/s | 593 kB 00:00 2023-07-29T13:25:59.069 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:25:59.072 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:25:59.072 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:25:59.115 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:25:59.116 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:25:59.301 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:25:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:25:59.426 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:25:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:59.686 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:25:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:25:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:25:59.687 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:25:59.800 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: python3-cephfs-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:25:59.909 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-cephfs-debuginfo 2023-07-29T13:26:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:26:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:00.097 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:26:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:00.098 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:26:00.328 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rados-debuginfo 2023-07-29T13:26:00.460 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:08:47 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:26:00.598 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:26:00.599 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:00.599 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:26:00.599 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:00.599 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:26:00.599 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 593 k 2023-07-29T13:26:00.600 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:00.600 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:26:00.600 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:00.600 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:26:00.600 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:00.600 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 593 k 2023-07-29T13:26:00.600 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 2.0 M 2023-07-29T13:26:00.601 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:26:00.874 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:08:46 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:26:01.010 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:26:01.011 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:01.011 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:26:01.011 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:01.012 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:26:01.012 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 769 k 2023-07-29T13:26:01.012 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:01.012 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:26:01.013 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:01.013 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:26:01.013 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:01.014 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 769 k 2023-07-29T13:26:01.014 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 2.7 M 2023-07-29T13:26:01.014 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:26:01.586 INFO:teuthology.orchestra.run.smithi040.stdout:python3-cephfs-debuginfo-18.1.3-11.g0484097b.el 601 kB/s | 593 kB 00:00 2023-07-29T13:26:01.587 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:01.588 INFO:teuthology.orchestra.run.smithi040.stdout:Total 600 kB/s | 593 kB 00:00 2023-07-29T13:26:01.588 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:26:01.592 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:26:01.592 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:26:01.637 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:26:01.637 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:26:01.823 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:26:01.911 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:26:02.046 INFO:teuthology.orchestra.run.smithi045.stdout:python3-rados-debuginfo-18.1.3-11.g0484097b.el9 743 kB/s | 769 kB 00:01 2023-07-29T13:26:02.047 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:02.047 INFO:teuthology.orchestra.run.smithi045.stdout:Total 742 kB/s | 769 kB 00:01 2023-07-29T13:26:02.048 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:26:02.051 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:26:02.052 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:26:02.099 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:26:02.099 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:26:02.288 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:26:02.310 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: python3-cephfs-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:26:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-debuginfo-2:18.1.3-11.g0484097b.el9.x8 1/1 2023-07-29T13:26:02.579 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-debuginfo-2:18.1.3-11.g0484097b.el9.x 1/1 2023-07-29T13:26:02.579 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:02.579 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:26:02.580 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:02.580 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:02.580 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:26:02.809 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rados-debuginfo 2023-07-29T13:26:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: python3-rados-debuginfo-2:18.1.3-11.g0484097b.el9.x8 1/1 2023-07-29T13:26:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-debuginfo-2:18.1.3-11.g0484097b.el9.x8 1/1 2023-07-29T13:26:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:03.109 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:26:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:03.110 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:26:03.347 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:08:50 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:26:03.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rbd-debuginfo 2023-07-29T13:26:03.483 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:26:03.484 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:03.484 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:26:03.484 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:03.484 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:26:03.484 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 769 k 2023-07-29T13:26:03.485 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:03.485 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:26:03.485 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:03.485 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:26:03.485 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:03.485 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 769 k 2023-07-29T13:26:03.485 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 2.7 M 2023-07-29T13:26:03.485 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:26:03.907 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:08:49 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:26:03.990 INFO:teuthology.orchestra.run.smithi040.stdout:python3-rados-debuginfo-18.1.3-11.g0484097b.el9 1.5 MB/s | 769 kB 00:00 2023-07-29T13:26:03.991 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:03.991 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.5 MB/s | 769 kB 00:00 2023-07-29T13:26:03.991 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:26:03.994 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:26:03.995 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:26:04.036 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:26:04.036 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:26:04.048 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:26:04.049 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:26:04.049 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:04.049 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:26:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 1.1 M 2023-07-29T13:26:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:04.050 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:26:04.050 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:04.050 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:26:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:04.051 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 1.1 M 2023-07-29T13:26:04.051 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 4.3 M 2023-07-29T13:26:04.051 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:26:04.239 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:26:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-debuginfo-2:18.1.3-11.g0484097b.el9.x8 1/1 2023-07-29T13:26:04.742 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: python3-rados-debuginfo-2:18.1.3-11.g0484097b.el9.x8 1/1 2023-07-29T13:26:05.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-debuginfo-2:18.1.3-11.g0484097b.el9.x8 1/1 2023-07-29T13:26:05.012 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:05.012 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:26:05.012 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:05.012 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:05.012 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:26:05.133 INFO:teuthology.orchestra.run.smithi045.stdout:python3-rbd-debuginfo-18.1.3-11.g0484097b.el9.x 1.0 MB/s | 1.1 MB 00:01 2023-07-29T13:26:05.133 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:05.134 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.0 MB/s | 1.1 MB 00:01 2023-07-29T13:26:05.134 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:26:05.138 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:26:05.138 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:26:05.186 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:26:05.186 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:26:05.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rbd-debuginfo 2023-07-29T13:26:05.427 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:26:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:05.827 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:08:52 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:26:05.964 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:26:05.965 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:05.966 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:26:05.966 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:05.966 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:26:05.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 1.1 M 2023-07-29T13:26:05.966 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:05.966 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:26:05.966 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:05.967 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:26:05.967 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:05.967 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 1.1 M 2023-07-29T13:26:05.967 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 4.3 M 2023-07-29T13:26:05.967 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:26:05.991 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: python3-rbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:06.291 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:06.292 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:06.292 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:26:06.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:06.292 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:06.292 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:26:06.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rgw-debuginfo 2023-07-29T13:26:06.677 INFO:teuthology.orchestra.run.smithi040.stdout:python3-rbd-debuginfo-18.1.3-11.g0484097b.el9.x 1.6 MB/s | 1.1 MB 00:00 2023-07-29T13:26:06.678 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:06.678 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.6 MB/s | 1.1 MB 00:00 2023-07-29T13:26:06.678 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:26:06.682 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:26:06.682 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:26:06.726 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:26:06.727 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:26:06.963 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:26:07.023 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:08:53 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:26:07.060 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:07.158 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:26:07.159 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:07.159 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:26:07.159 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:07.159 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:26:07.159 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 337 k 2023-07-29T13:26:07.159 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:07.159 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:26:07.160 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:07.160 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:26:07.160 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:07.160 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 337 k 2023-07-29T13:26:07.160 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 1.0 M 2023-07-29T13:26:07.160 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:26:07.570 INFO:teuthology.orchestra.run.smithi045.stdout:python3-rgw-debuginfo-18.1.3-11.g0484097b.el9.x 823 kB/s | 337 kB 00:00 2023-07-29T13:26:07.570 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:07.571 INFO:teuthology.orchestra.run.smithi045.stdout:Total 820 kB/s | 337 kB 00:00 2023-07-29T13:26:07.571 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:26:07.575 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:26:07.575 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:26:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: python3-rbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:07.614 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:26:07.614 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:26:07.756 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:26:07.870 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:07.870 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:07.870 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:07.871 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:26:07.871 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:07.871 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:07.871 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:26:08.100 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rgw-debuginfo 2023-07-29T13:26:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: python3-rgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:08.546 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:08.546 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:08.546 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:26:08.546 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:08.547 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:08.547 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:26:08.651 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:08:55 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:26:08.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-fuse-debuginfo 2023-07-29T13:26:08.787 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:26:08.788 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:08.788 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:26:08.788 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:08.788 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:26:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 337 k 2023-07-29T13:26:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:08.789 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:26:08.789 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:08.789 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:26:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:08.790 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 337 k 2023-07-29T13:26:08.790 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 1.0 M 2023-07-29T13:26:08.790 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:26:09.197 INFO:teuthology.orchestra.run.smithi040.stdout:python3-rgw-debuginfo-18.1.3-11.g0484097b.el9.x 827 kB/s | 337 kB 00:00 2023-07-29T13:26:09.198 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:09.198 INFO:teuthology.orchestra.run.smithi040.stdout:Total 824 kB/s | 337 kB 00:00 2023-07-29T13:26:09.198 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:26:09.202 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:26:09.202 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:26:09.240 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:26:09.241 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:26:09.290 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:08:55 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:26:09.384 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:26:09.427 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:26:09.428 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:09.429 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:26:09.429 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:09.429 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:26:09.429 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 507 k 2023-07-29T13:26:09.429 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:09.429 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:26:09.429 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:09.429 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:26:09.430 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:09.430 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 507 k 2023-07-29T13:26:09.430 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 2.5 M 2023-07-29T13:26:09.430 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:26:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:09.878 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: python3-rgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:09.956 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-fuse-debuginfo-18.1.3-11.g0484097b.el9.x86_ 962 kB/s | 507 kB 00:00 2023-07-29T13:26:09.957 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:09.957 INFO:teuthology.orchestra.run.smithi045.stdout:Total 959 kB/s | 507 kB 00:00 2023-07-29T13:26:09.957 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:26:09.961 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:26:09.961 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:26:10.001 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:26:10.001 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:26:10.167 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86_ 1/1 2023-07-29T13:26:10.168 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:10.168 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:26:10.168 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:10.168 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:10.168 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:26:10.170 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:26:10.306 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:10.389 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-fuse-debuginfo 2023-07-29T13:26:10.701 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:10.935 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:08:57 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:26:10.983 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:10.984 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:10.984 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:26:10.984 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:10.984 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:10.984 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:26:11.072 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:26:11.073 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:11.073 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:26:11.073 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:11.074 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:26:11.074 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 507 k 2023-07-29T13:26:11.074 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:11.074 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:26:11.074 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:11.075 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:26:11.075 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:11.075 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 507 k 2023-07-29T13:26:11.075 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 2.5 M 2023-07-29T13:26:11.075 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:26:11.208 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-mirror-debuginfo 2023-07-29T13:26:11.668 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-fuse-debuginfo-18.1.3-11.g0484097b.el9.x86_ 854 kB/s | 507 kB 00:00 2023-07-29T13:26:11.668 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:11.669 INFO:teuthology.orchestra.run.smithi040.stdout:Total 852 kB/s | 507 kB 00:00 2023-07-29T13:26:11.669 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:26:11.672 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:26:11.672 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:26:11.713 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:26:11.714 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:26:11.761 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:08:57 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:26:11.885 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:26:11.899 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:26:11.900 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:11.900 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:26:11.900 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:11.900 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:26:11.901 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 51 M 2023-07-29T13:26:11.901 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:11.901 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:26:11.901 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:11.901 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:26:11.901 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:11.901 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 51 M 2023-07-29T13:26:11.901 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 282 M 2023-07-29T13:26:11.902 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:26:12.001 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:12.420 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:12.686 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:12.687 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:12.687 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:26:12.687 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:12.687 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:12.687 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:26:12.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-mirror-debuginfo 2023-07-29T13:26:13.445 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:09:00 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:26:13.580 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:26:13.581 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:13.581 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:26:13.581 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:13.582 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:26:13.582 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 51 M 2023-07-29T13:26:13.582 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:13.582 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:26:13.582 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:13.582 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:26:13.582 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:13.583 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 51 M 2023-07-29T13:26:13.583 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 282 M 2023-07-29T13:26:13.583 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:26:15.984 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-mirror-debuginfo-18.1.3-11.g0484097b.el9.x8 12 MB/s | 51 MB 00:04 2023-07-29T13:26:15.985 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:15.986 INFO:teuthology.orchestra.run.smithi045.stdout:Total 12 MB/s | 51 MB 00:04 2023-07-29T13:26:15.986 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:26:15.990 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:26:15.990 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:26:16.341 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:26:16.341 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:26:17.979 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-mirror-debuginfo-18.1.3-11.g0484097b.el9.x8 12 MB/s | 51 MB 00:04 2023-07-29T13:26:17.980 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:17.980 INFO:teuthology.orchestra.run.smithi040.stdout:Total 12 MB/s | 51 MB 00:04 2023-07-29T13:26:17.981 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:26:17.985 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:26:17.985 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:26:18.335 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:26:18.335 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:26:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:26:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:26:24.436 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:26:24.527 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:26:24.543 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:26:24.816 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:26:24.816 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:24.817 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:26:24.817 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:24.817 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:24.817 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:26:25.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-nbd-debuginfo 2023-07-29T13:26:25.564 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:09:11 ago on Sat 29 Jul 2023 01:17:14 PM UTC. 2023-07-29T13:26:25.700 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-07-29T13:26:25.701 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:25.701 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-07-29T13:26:25.701 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:25.701 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-07-29T13:26:25.701 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 1.2 M 2023-07-29T13:26:25.701 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:25.702 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-07-29T13:26:25.702 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-07-29T13:26:25.702 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2023-07-29T13:26:25.702 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:25.702 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 1.2 M 2023-07-29T13:26:25.702 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 5.4 M 2023-07-29T13:26:25.702 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-07-29T13:26:26.362 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-nbd-debuginfo-18.1.3-11.g0484097b.el9.x86_6 1.9 MB/s | 1.2 MB 00:00 2023-07-29T13:26:26.363 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:26.363 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.9 MB/s | 1.2 MB 00:00 2023-07-29T13:26:26.363 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-07-29T13:26:26.367 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-07-29T13:26:26.367 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-07-29T13:26:26.411 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-07-29T13:26:26.412 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-07-29T13:26:26.561 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:26:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-07-29T13:26:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:26.832 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-debuginfo-2:18.1.3-11.g0484097b.el9.x86_6 1/1 2023-07-29T13:26:26.833 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:26.833 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:26:26.833 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:26.833 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:26.833 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:26:27.028 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-nbd-debuginfo 2023-07-29T13:26:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-nbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:27.515 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-07-29T13:26:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:27.515 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-07-29T13:26:27.572 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:09:14 ago on Sat 29 Jul 2023 01:17:13 PM UTC. 2023-07-29T13:26:27.704 DEBUG:teuthology.parallel:result is None 2023-07-29T13:26:27.709 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-29T13:26:27.710 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2023-07-29T13:26:27.710 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:27.710 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-29T13:26:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-debuginfo x86_64 2:18.1.3-11.g0484097b.el9 ceph 1.2 M 2023-07-29T13:26:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:27.711 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-29T13:26:27.711 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-29T13:26:27.711 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2023-07-29T13:26:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:27.712 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 1.2 M 2023-07-29T13:26:27.712 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 5.4 M 2023-07-29T13:26:27.712 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-29T13:26:28.615 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-nbd-debuginfo-18.1.3-11.g0484097b.el9.x86_6 1.4 MB/s | 1.2 MB 00:00 2023-07-29T13:26:28.615 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-29T13:26:28.616 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.4 MB/s | 1.2 MB 00:00 2023-07-29T13:26:28.616 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-29T13:26:28.620 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-29T13:26:28.620 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-29T13:26:28.666 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-29T13:26:28.666 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-29T13:26:28.912 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-29T13:26:29.040 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:29.463 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-nbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 1/1 2023-07-29T13:26:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:29.751 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-29T13:26:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-debuginfo-2:18.1.3-11.g0484097b.el9.x86_64 2023-07-29T13:26:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:29.752 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-29T13:26:29.958 DEBUG:teuthology.parallel:result is None 2023-07-29T13:26:29.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0484097b0572e96f42cf6402cbe0ac8dcb046577 2023-07-29T13:26:30.197 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-29T13:26:30.218 INFO:teuthology.orchestra.run.smithi040.stdout:18.1.3-11.g0484097b.el9 2023-07-29T13:26:30.219 INFO:teuthology.packaging:The installed version of ceph is 18.1.3-11.g0484097b.el9 2023-07-29T13:26:30.219 INFO:teuthology.task.install:The correct ceph version 18.1.3-11.g0484097b is installed. 2023-07-29T13:26:30.219 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0484097b0572e96f42cf6402cbe0ac8dcb046577 2023-07-29T13:26:30.441 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-29T13:26:30.466 INFO:teuthology.orchestra.run.smithi045.stdout:18.1.3-11.g0484097b.el9 2023-07-29T13:26:30.466 INFO:teuthology.packaging:The installed version of ceph is 18.1.3-11.g0484097b.el9 2023-07-29T13:26:30.466 INFO:teuthology.task.install:The correct ceph version 18.1.3-11.g0484097b is installed. 2023-07-29T13:26:30.467 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-29T13:26:30.467 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:26:30.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-29T13:26:30.500 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:26:30.500 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-29T13:26:30.540 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-29T13:26:30.541 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:26:30.541 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2023-07-29T13:26:30.573 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-29T13:26:30.643 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:26:30.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2023-07-29T13:26:30.678 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-29T13:26:30.750 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-29T13:26:30.751 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:26:30.751 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-29T13:26:30.782 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-29T13:26:30.853 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:26:30.854 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-29T13:26:30.886 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-29T13:26:30.958 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-29T13:26:30.958 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:26:30.959 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2023-07-29T13:26:30.990 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-29T13:26:31.060 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:26:31.061 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2023-07-29T13:26:31.094 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-29T13:26:31.170 INFO:teuthology.run_tasks:Running task ceph... 2023-07-29T13:26:31.257 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-07-29T13:26:31.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2023-07-29T13:26:31.260 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /var/log/ceph 2023-07-29T13:26:31.291 INFO:tasks.ceph:Disabling ceph logrotate... 2023-07-29T13:26:31.292 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-29T13:26:31.331 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-29T13:26:31.363 INFO:tasks.ceph:Creating extra log directories... 2023-07-29T13:26:31.363 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-29T13:26:31.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-29T13:26:31.445 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-07-29T13:26:31.445 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-07-29T13:26:31.445 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625', 'branch': 'reef-release', 'description': 'rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7356625', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'reef-release'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': '0484097b0572e96f42cf6402cbe0ac8dcb046577', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '0484097b0572e96f42cf6402cbe0ac8dcb046577'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef-release', 'sha1': '0484097b0572e96f42cf6402cbe0ac8dcb046577'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5363, 'sha1': '0484097b0572e96f42cf6402cbe0ac8dcb046577', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'reef-release', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '0484097b0572e96f42cf6402cbe0ac8dcb046577', 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMq9L1pCZlICkqLKsCQ1cbsTIVQukPVXnoChMPUjBzoDV3riqmHBm32FQWVTGmE7j0ZBfq5L6k7As7l0oYgfcqE=', 'smithi045.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRiuvrnGNaq6R3f1uQMQUyTEeSjQdZBYrRH8yP1hs8qaTIr5bJ8hekyG39aJbWdoGjv81vqMy/3vTr8gC0vGJA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'workunit': {'clients': {'client.0': ['rados/test.sh']}, 'env': {'ALLOW_TIMEOUTS': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '61140164d2579c30ffff3970615e0f82e40c6470', 'timestamp': '2023-07-28_23:11:59', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-07-29T13:26:31.446 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-29T13:26:31.473 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-29T13:26:31.489 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2023-07-29T13:26:31.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/run/ceph 2023-07-29T13:26:31.559 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:26:31.560 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2023-07-29T13:26:31.612 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-29T13:26:31.613 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2023-07-29T13:26:31.668 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-29T13:26:31.668 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-29T13:26:31.669 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 655 Links: 1 2023-07-29T13:26:31.669 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-29T13:26:31.669 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-07-29T13:26:31.670 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-07-29 13:19:05.367139782 +0000 2023-07-29T13:26:31.670 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-07-29 13:16:26.116895721 +0000 2023-07-29T13:26:31.670 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-07-29 13:16:26.116895721 +0000 2023-07-29T13:26:31.670 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2023-07-29 13:16:26.116895721 +0000 2023-07-29T13:26:31.671 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-29T13:26:31.737 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-07-29T13:26:31.737 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-07-29T13:26:31.737 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000207207 s, 2.5 MB/s 2023-07-29T13:26:31.738 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-29T13:26:31.794 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2023-07-29T13:26:31.849 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-29T13:26:31.850 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-29T13:26:31.850 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 672 Links: 1 2023-07-29T13:26:31.850 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-29T13:26:31.850 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-07-29T13:26:31.850 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-07-29 13:19:05.367139782 +0000 2023-07-29T13:26:31.850 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-07-29 13:16:26.625890117 +0000 2023-07-29T13:26:31.851 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-07-29 13:16:26.625890117 +0000 2023-07-29T13:26:31.851 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2023-07-29 13:16:26.625890117 +0000 2023-07-29T13:26:31.851 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-29T13:26:31.918 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-07-29T13:26:31.918 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-07-29T13:26:31.919 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000205101 s, 2.5 MB/s 2023-07-29T13:26:31.920 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-29T13:26:31.975 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2023-07-29T13:26:32.031 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-29T13:26:32.031 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-29T13:26:32.031 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 689 Links: 1 2023-07-29T13:26:32.031 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-29T13:26:32.031 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-07-29T13:26:32.032 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-07-29 13:19:05.368139771 +0000 2023-07-29T13:26:32.032 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-07-29 13:16:27.132884535 +0000 2023-07-29T13:26:32.032 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-07-29 13:16:27.132884535 +0000 2023-07-29T13:26:32.032 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2023-07-29 13:16:27.132884535 +0000 2023-07-29T13:26:32.032 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-29T13:26:32.099 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-07-29T13:26:32.099 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-07-29T13:26:32.100 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.00021313 s, 2.4 MB/s 2023-07-29T13:26:32.101 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-29T13:26:32.156 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2023-07-29T13:26:32.211 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-29T13:26:32.211 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-29T13:26:32.212 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 710 Links: 1 2023-07-29T13:26:32.212 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-29T13:26:32.212 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-07-29T13:26:32.212 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-07-29 13:19:05.368139771 +0000 2023-07-29T13:26:32.212 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-07-29 13:16:27.631879040 +0000 2023-07-29T13:26:32.212 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-07-29 13:16:27.631879040 +0000 2023-07-29T13:26:32.212 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2023-07-29 13:16:27.631879040 +0000 2023-07-29T13:26:32.213 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-29T13:26:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-07-29T13:26:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-07-29T13:26:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000176113 s, 2.9 MB/s 2023-07-29T13:26:32.282 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-29T13:26:32.337 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-29T13:26:32.337 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:26:32.338 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2023-07-29T13:26:32.356 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-29T13:26:32.357 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2023-07-29T13:26:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-29T13:26:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-29T13:26:32.417 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 654 Links: 1 2023-07-29T13:26:32.417 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-29T13:26:32.417 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-07-29T13:26:32.417 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-07-29 13:19:12.295007264 +0000 2023-07-29T13:26:32.418 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-07-29 13:16:16.666904727 +0000 2023-07-29T13:26:32.418 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-07-29 13:16:16.666904727 +0000 2023-07-29T13:26:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2023-07-29 13:16:16.666904727 +0000 2023-07-29T13:26:32.418 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-29T13:26:32.491 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-07-29T13:26:32.492 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-07-29T13:26:32.492 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000121898 s, 4.2 MB/s 2023-07-29T13:26:32.493 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-29T13:26:32.551 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2023-07-29T13:26:32.609 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-29T13:26:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-29T13:26:32.610 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 672 Links: 1 2023-07-29T13:26:32.610 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-29T13:26:32.610 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-07-29T13:26:32.610 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-07-29 13:19:12.295007264 +0000 2023-07-29T13:26:32.610 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-07-29 13:16:17.167897057 +0000 2023-07-29T13:26:32.611 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-07-29 13:16:17.167897057 +0000 2023-07-29T13:26:32.611 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2023-07-29 13:16:17.167897057 +0000 2023-07-29T13:26:32.611 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-29T13:26:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-07-29T13:26:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-07-29T13:26:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000116951 s, 4.4 MB/s 2023-07-29T13:26:32.685 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-29T13:26:32.743 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2023-07-29T13:26:32.801 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-29T13:26:32.801 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-29T13:26:32.801 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 691 Links: 1 2023-07-29T13:26:32.802 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-29T13:26:32.802 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-07-29T13:26:32.802 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-07-29 13:19:12.296007249 +0000 2023-07-29T13:26:32.802 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-07-29 13:16:17.675889278 +0000 2023-07-29T13:26:32.803 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-07-29 13:16:17.675889278 +0000 2023-07-29T13:26:32.803 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2023-07-29 13:16:17.675889278 +0000 2023-07-29T13:26:32.803 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-29T13:26:32.873 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-07-29T13:26:32.873 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-07-29T13:26:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000103475 s, 4.9 MB/s 2023-07-29T13:26:32.875 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-29T13:26:32.932 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2023-07-29T13:26:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-29T13:26:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-29T13:26:32.989 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 710 Links: 1 2023-07-29T13:26:32.989 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-29T13:26:32.990 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-07-29T13:26:32.990 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-07-29 13:19:12.296007249 +0000 2023-07-29T13:26:32.990 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-07-29 13:16:18.209881103 +0000 2023-07-29T13:26:32.990 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-07-29 13:16:18.209881103 +0000 2023-07-29T13:26:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2023-07-29 13:16:18.209881103 +0000 2023-07-29T13:26:32.990 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-29T13:26:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-07-29T13:26:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-07-29T13:26:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000108669 s, 4.7 MB/s 2023-07-29T13:26:33.062 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-29T13:26:33.119 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-29T13:26:33.120 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-29T13:26:33.120 INFO:tasks.ceph:Generating config... 2023-07-29T13:26:33.121 INFO:tasks.ceph:[client] debug monc = 20 2023-07-29T13:26:33.121 INFO:tasks.ceph:[client] debug ms = 1 2023-07-29T13:26:33.122 INFO:tasks.ceph:[client] debug objecter = 20 2023-07-29T13:26:33.122 INFO:tasks.ceph:[client] debug rados = 20 2023-07-29T13:26:33.122 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-07-29T13:26:33.122 INFO:tasks.ceph:[global] mon election default strategy = 1 2023-07-29T13:26:33.123 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-07-29T13:26:33.123 INFO:tasks.ceph:[global] ms type = async 2023-07-29T13:26:33.123 INFO:tasks.ceph:[global] osd heartbeat grace = 80 2023-07-29T13:26:33.123 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-07-29T13:26:33.123 INFO:tasks.ceph:[mgr] debug ms = 1 2023-07-29T13:26:33.124 INFO:tasks.ceph:[mon] debug mon = 20 2023-07-29T13:26:33.124 INFO:tasks.ceph:[mon] debug ms = 1 2023-07-29T13:26:33.124 INFO:tasks.ceph:[mon] debug paxos = 20 2023-07-29T13:26:33.124 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-07-29T13:26:33.125 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2023-07-29T13:26:33.125 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-07-29T13:26:33.125 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-07-29T13:26:33.125 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-07-29T13:26:33.126 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-07-29T13:26:33.126 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2023-07-29T13:26:33.126 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-07-29T13:26:33.126 INFO:tasks.ceph:[osd] bdev async discard = True 2023-07-29T13:26:33.126 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-07-29T13:26:33.127 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-07-29T13:26:33.127 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-07-29T13:26:33.128 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-07-29T13:26:33.128 INFO:tasks.ceph:[osd] debug bluefs = 1 2023-07-29T13:26:33.129 INFO:tasks.ceph:[osd] debug bluestore = 1 2023-07-29T13:26:33.129 INFO:tasks.ceph:[osd] debug monc = 20 2023-07-29T13:26:33.130 INFO:tasks.ceph:[osd] debug ms = 1 2023-07-29T13:26:33.130 INFO:tasks.ceph:[osd] debug osd = 20 2023-07-29T13:26:33.131 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-07-29T13:26:33.132 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-07-29T13:26:33.132 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-07-29T13:26:33.132 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-07-29T13:26:33.132 INFO:tasks.ceph:[osd] osd class default list = * 2023-07-29T13:26:33.133 INFO:tasks.ceph:[osd] osd class load list = * 2023-07-29T13:26:33.133 INFO:tasks.ceph:[osd] osd client watch timeout = 120 2023-07-29T13:26:33.133 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-07-29T13:26:33.133 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-07-29T13:26:33.133 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-07-29T13:26:33.134 INFO:tasks.ceph:[osd] osd fast shutdown = False 2023-07-29T13:26:33.134 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-07-29T13:26:33.134 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2023-07-29T13:26:33.134 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2023-07-29T13:26:33.135 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-07-29T13:26:33.135 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-07-29T13:26:33.135 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-07-29T13:26:33.135 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-07-29T13:26:33.135 INFO:tasks.ceph:Setting up mon.a... 2023-07-29T13:26:33.136 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-07-29T13:26:33.199 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2023-07-29T13:26:33.202 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-07-29T13:26:33.253 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-29T13:26:33.284 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.c', '[v2:172.21.15.40:3301,v1:172.21.15.40:6790]'), ('mon.b', '172.21.15.45')] 2023-07-29T13:26:33.284 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,[v2:172.21.15.40:3301,v1:172.21.15.40:6790],172.21.15.45', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-07-29T13:26:33.285 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:26:33.286 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-07-29T13:26:33.339 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --addv c '[v2:172.21.15.40:3301,v1:172.21.15.40:6790]' --add b 172.21.15.45 --print /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:33.424 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:33.424 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:26:33.424 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = pacific 2023-07-29T13:26:33.425 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2023-07-29T13:26:33.425 INFO:teuthology.orchestra.run.smithi040.stdout:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:26:33.425 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:26:33.425 INFO:teuthology.orchestra.run.smithi040.stdout:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:26:33.425 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 16 (pacific) 2023-07-29T13:26:33.425 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2023-07-29T13:26:33.425 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:26:33.425 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:26:33.426 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:26:33.426 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-07-29T13:26:33.427 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-07-29T13:26:33.441 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d11937c2-52a1-49ac-aadc-1ae254eda669... 2023-07-29T13:26:33.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-29T13:26:33.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-29T13:26:33.542 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2023-07-29T13:26:33.542 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2023-07-29T13:26:33.542 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-29T13:26:33.543 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2023-07-29T13:26:33.543 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.543 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2023-07-29T13:26:33.543 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.543 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2023-07-29T13:26:33.543 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.543 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2023-07-29T13:26:33.544 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2023-07-29T13:26:33.544 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.544 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2023-07-29T13:26:33.544 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2023-07-29T13:26:33.544 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.544 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-29T13:26:33.545 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2023-07-29T13:26:33.545 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.545 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2023-07-29T13:26:33.545 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-29T13:26:33.545 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.545 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2023-07-29T13:26:33.545 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.546 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2023-07-29T13:26:33.546 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2023-07-29T13:26:33.546 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2023-07-29T13:26:33.546 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2023-07-29T13:26:33.546 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2023-07-29T13:26:33.546 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2023-07-29T13:26:33.547 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2023-07-29T13:26:33.547 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-29T13:26:33.547 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-29T13:26:33.547 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2023-07-29T13:26:33.547 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.547 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-29T13:26:33.547 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.548 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2023-07-29T13:26:33.548 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.548 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2023-07-29T13:26:33.548 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.549 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2023-07-29T13:26:33.549 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2023-07-29T13:26:33.549 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2023-07-29T13:26:33.549 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,[v2:172.21.15.40:3301,v1:172.21.15.40:6790],172.21.15.45" 2023-07-29T13:26:33.549 INFO:teuthology.orchestra.run.smithi040.stdout: mon client directed command retry = 5 2023-07-29T13:26:33.550 INFO:teuthology.orchestra.run.smithi040.stdout: mon election default strategy = 1 2023-07-29T13:26:33.550 INFO:teuthology.orchestra.run.smithi040.stdout: ms inject socket failures = 5000 2023-07-29T13:26:33.550 INFO:teuthology.orchestra.run.smithi040.stdout: ms type = async 2023-07-29T13:26:33.550 INFO:teuthology.orchestra.run.smithi040.stdout: osd heartbeat grace = 80 2023-07-29T13:26:33.550 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:26:33.550 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.550 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2023-07-29T13:26:33.551 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2023-07-29T13:26:33.551 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.551 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2023-07-29T13:26:33.551 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2023-07-29T13:26:33.551 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2023-07-29T13:26:33.551 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.551 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2023-07-29T13:26:33.552 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2023-07-29T13:26:33.552 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.552 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2023-07-29T13:26:33.552 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2023-07-29T13:26:33.552 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2023-07-29T13:26:33.552 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.553 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2023-07-29T13:26:33.553 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2023-07-29T13:26:33.553 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.553 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2023-07-29T13:26:33.554 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.554 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2023-07-29T13:26:33.554 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.554 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2023-07-29T13:26:33.555 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.555 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2023-07-29T13:26:33.555 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2023-07-29T13:26:33.555 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.555 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2023-07-29T13:26:33.555 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2023-07-29T13:26:33.556 INFO:teuthology.orchestra.run.smithi040.stdout: bdev async discard = True 2023-07-29T13:26:33.556 INFO:teuthology.orchestra.run.smithi040.stdout: bdev enable discard = True 2023-07-29T13:26:33.556 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2023-07-29T13:26:33.556 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2023-07-29T13:26:33.556 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore zero block detection = True 2023-07-29T13:26:33.556 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 1 2023-07-29T13:26:33.557 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 1 2023-07-29T13:26:33.557 INFO:teuthology.orchestra.run.smithi040.stdout: debug monc = 20 2023-07-29T13:26:33.557 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-07-29T13:26:33.557 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2023-07-29T13:26:33.557 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 10 2023-07-29T13:26:33.557 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2023-07-29T13:26:33.557 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2023-07-29T13:26:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2023-07-29T13:26:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: osd class default list = * 2023-07-29T13:26:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: osd class load list = * 2023-07-29T13:26:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: osd client watch timeout = 120 2023-07-29T13:26:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify cached snaps = True 2023-07-29T13:26:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify missing on start = True 2023-07-29T13:26:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2023-07-29T13:26:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: osd fast shutdown = False 2023-07-29T13:26:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock override recovery settings = True 2023-07-29T13:26:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: osd memory target = 1610612736 2023-07-29T13:26:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2023-07-29T13:26:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: osd op queue = debug_random 2023-07-29T13:26:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: osd op queue cut off = debug_random 2023-07-29T13:26:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: osd shutdown pgref assert = True 2023-07-29T13:26:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.560 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2023-07-29T13:26:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-07-29T13:26:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2023-07-29T13:26:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2023-07-29T13:26:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2023-07-29T13:26:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2023-07-29T13:26:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2023-07-29T13:26:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2023-07-29T13:26:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.561 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2023-07-29T13:26:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-07-29T13:26:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2023-07-29T13:26:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2023-07-29T13:26:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2023-07-29T13:26:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2023-07-29T13:26:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2023-07-29T13:26:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2023-07-29T13:26:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2023-07-29T13:26:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2023-07-29T13:26:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2023-07-29T13:26:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-29T13:26:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2023-07-29T13:26:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2023-07-29T13:26:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2023-07-29T13:26:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-29T13:26:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-29T13:26:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2023-07-29T13:26:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2023-07-29T13:26:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2023-07-29T13:26:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: mon min osdmap epochs = 50 2023-07-29T13:26:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd crush smoke test = False 2023-07-29T13:26:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: mon osdmap full prune interval = 2 2023-07-29T13:26:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: mon osdmap full prune min = 15 2023-07-29T13:26:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: mon osdmap full prune txsize = 2 2023-07-29T13:26:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: mon scrub interval = 300 2023-07-29T13:26:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on pool no app = False 2023-07-29T13:26:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: paxos service trim min = 10 2023-07-29T13:26:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:26:33.566 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2023-07-29T13:26:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2023-07-29T13:26:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2023-07-29T13:26:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2023-07-29T13:26:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-29T13:26:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-29T13:26:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: debug monc = 20 2023-07-29T13:26:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-07-29T13:26:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: debug objecter = 20 2023-07-29T13:26:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: debug rados = 20 2023-07-29T13:26:33.567 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2023-07-29T13:26:33.568 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2023-07-29T13:26:33.568 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2023-07-29T13:26:33.568 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2023-07-29T13:26:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2023-07-29T13:26:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-29T13:26:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2023-07-29T13:26:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2023-07-29T13:26:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2023-07-29T13:26:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2023-07-29T13:26:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2023-07-29T13:26:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2023-07-29T13:26:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2023-07-29T13:26:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-29T13:26:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2023-07-29T13:26:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2023-07-29T13:26:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-29T13:26:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2023-07-29T13:26:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2023-07-29T13:26:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2023-07-29T13:26:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2023-07-29T13:26:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2023-07-29T13:26:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2023-07-29T13:26:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2023-07-29T13:26:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2023-07-29T13:26:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-29T13:26:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-29T13:26:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2023-07-29T13:26:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-29T13:26:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2023-07-29T13:26:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2023-07-29T13:26:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2023-07-29T13:26:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2023-07-29T13:26:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2023-07-29T13:26:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.40,[v2:172.21.15.40:3301,v1:172.21.15.40:6790],172.21.15.45" 2023-07-29T13:26:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: mon client directed command retry = 5 2023-07-29T13:26:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: mon election default strategy = 1 2023-07-29T13:26:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: ms inject socket failures = 5000 2023-07-29T13:26:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: ms type = async 2023-07-29T13:26:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: osd heartbeat grace = 80 2023-07-29T13:26:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:26:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.576 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2023-07-29T13:26:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2023-07-29T13:26:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2023-07-29T13:26:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2023-07-29T13:26:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2023-07-29T13:26:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2023-07-29T13:26:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2023-07-29T13:26:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2023-07-29T13:26:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2023-07-29T13:26:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2023-07-29T13:26:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2023-07-29T13:26:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2023-07-29T13:26:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2023-07-29T13:26:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2023-07-29T13:26:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2023-07-29T13:26:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2023-07-29T13:26:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2023-07-29T13:26:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2023-07-29T13:26:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2023-07-29T13:26:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: bdev async discard = True 2023-07-29T13:26:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: bdev enable discard = True 2023-07-29T13:26:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2023-07-29T13:26:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2023-07-29T13:26:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore zero block detection = True 2023-07-29T13:26:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 1 2023-07-29T13:26:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 1 2023-07-29T13:26:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: debug monc = 20 2023-07-29T13:26:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-07-29T13:26:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2023-07-29T13:26:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 10 2023-07-29T13:26:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2023-07-29T13:26:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2023-07-29T13:26:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2023-07-29T13:26:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: osd class default list = * 2023-07-29T13:26:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: osd class load list = * 2023-07-29T13:26:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: osd client watch timeout = 120 2023-07-29T13:26:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify cached snaps = True 2023-07-29T13:26:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify missing on start = True 2023-07-29T13:26:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2023-07-29T13:26:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: osd fast shutdown = False 2023-07-29T13:26:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock override recovery settings = True 2023-07-29T13:26:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: osd memory target = 1610612736 2023-07-29T13:26:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2023-07-29T13:26:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: osd op queue = debug_random 2023-07-29T13:26:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: osd op queue cut off = debug_random 2023-07-29T13:26:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: osd shutdown pgref assert = True 2023-07-29T13:26:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.584 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2023-07-29T13:26:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-07-29T13:26:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2023-07-29T13:26:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2023-07-29T13:26:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2023-07-29T13:26:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2023-07-29T13:26:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2023-07-29T13:26:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2023-07-29T13:26:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.585 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2023-07-29T13:26:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-07-29T13:26:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2023-07-29T13:26:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2023-07-29T13:26:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2023-07-29T13:26:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2023-07-29T13:26:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2023-07-29T13:26:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2023-07-29T13:26:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2023-07-29T13:26:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2023-07-29T13:26:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2023-07-29T13:26:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-29T13:26:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2023-07-29T13:26:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2023-07-29T13:26:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2023-07-29T13:26:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-29T13:26:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-29T13:26:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2023-07-29T13:26:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2023-07-29T13:26:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2023-07-29T13:26:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: mon min osdmap epochs = 50 2023-07-29T13:26:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd crush smoke test = False 2023-07-29T13:26:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: mon osdmap full prune interval = 2 2023-07-29T13:26:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: mon osdmap full prune min = 15 2023-07-29T13:26:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: mon osdmap full prune txsize = 2 2023-07-29T13:26:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: mon scrub interval = 300 2023-07-29T13:26:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on pool no app = False 2023-07-29T13:26:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: paxos service trim min = 10 2023-07-29T13:26:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-07-29T13:26:33.590 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2023-07-29T13:26:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2023-07-29T13:26:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2023-07-29T13:26:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2023-07-29T13:26:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-29T13:26:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-29T13:26:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: debug monc = 20 2023-07-29T13:26:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-07-29T13:26:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: debug objecter = 20 2023-07-29T13:26:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: debug rados = 20 2023-07-29T13:26:33.591 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2023-07-29T13:26:33.591 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2023-07-29T13:26:33.591 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2023-07-29T13:26:33.593 INFO:tasks.ceph:Creating admin key on mon.a... 2023-07-29T13:26:33.593 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-07-29T13:26:33.653 INFO:tasks.ceph:Copying monmap to all nodes... 2023-07-29T13:26:33.654 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:26:33.654 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-07-29T13:26:33.670 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:26:33.671 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-07-29T13:26:33.725 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2023-07-29T13:26:33.726 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:26:33.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-29T13:26:33.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-29T13:26:33.811 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:26:33.811 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:33.866 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi045.front.sepia.ceph.com 2023-07-29T13:26:33.867 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:26:33.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-29T13:26:33.868 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-29T13:26:33.920 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:26:33.920 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:33.977 INFO:tasks.ceph:Setting up mon nodes... 2023-07-29T13:26:33.977 INFO:tasks.ceph:Setting up mgr nodes... 2023-07-29T13:26:33.978 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-07-29T13:26:34.041 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-07-29T13:26:34.044 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-07-29T13:26:34.129 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-07-29T13:26:34.131 INFO:tasks.ceph:Setting up mds nodes... 2023-07-29T13:26:34.132 INFO:tasks.ceph_client:Setting up client nodes... 2023-07-29T13:26:34.132 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-29T13:26:34.179 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-07-29T13:26:34.197 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-07-29T13:26:34.245 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-07-29T13:26:34.264 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-07-29T13:26:34.264 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-29T13:26:34.265 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-07-29T13:26:34.295 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-29T13:26:34.296 INFO:tasks.ceph:role: osd.0 2023-07-29T13:26:34.296 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2023-07-29T13:26:34.297 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-29T13:26:34.373 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-29T13:26:34.373 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-29T13:26:34.373 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-29T13:26:34.374 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-07-29T13:26:34.374 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-29T13:26:34.374 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-07-29T13:26:34.374 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-29T13:26:34.374 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-29T13:26:34.374 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-29T13:26:34.374 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-29T13:26:34.853 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-07-29T13:26:34.854 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-07-29T13:26:34.855 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-07-29T13:26:34.982 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-07-29T13:26:35.021 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-07-29T13:26:35.094 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-29T13:26:35.094 INFO:tasks.ceph:role: osd.1 2023-07-29T13:26:35.094 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2023-07-29T13:26:35.095 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-29T13:26:35.173 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-29T13:26:35.173 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-29T13:26:35.173 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-29T13:26:35.173 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-07-29T13:26:35.174 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-29T13:26:35.174 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-07-29T13:26:35.174 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-29T13:26:35.174 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-29T13:26:35.174 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-29T13:26:35.174 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-29T13:26:35.600 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-07-29T13:26:35.602 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-07-29T13:26:35.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-07-29T13:26:35.703 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-07-29T13:26:35.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-07-29T13:26:35.811 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-29T13:26:35.811 INFO:tasks.ceph:role: osd.2 2023-07-29T13:26:35.812 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2023-07-29T13:26:35.812 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-29T13:26:35.891 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-29T13:26:35.891 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-29T13:26:35.891 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-29T13:26:35.892 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-07-29T13:26:35.892 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-29T13:26:35.892 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-07-29T13:26:35.892 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-29T13:26:35.892 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-29T13:26:35.892 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-29T13:26:35.892 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-29T13:26:36.352 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-07-29T13:26:36.353 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-07-29T13:26:36.354 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-07-29T13:26:36.485 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-07-29T13:26:36.524 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-07-29T13:26:36.596 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-29T13:26:36.597 INFO:tasks.ceph:role: osd.3 2023-07-29T13:26:36.597 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2023-07-29T13:26:36.597 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-29T13:26:36.676 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-29T13:26:36.677 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-29T13:26:36.677 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-29T13:26:36.677 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-07-29T13:26:36.677 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-29T13:26:36.677 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-07-29T13:26:36.677 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-29T13:26:36.677 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-29T13:26:36.678 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-29T13:26:36.678 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-29T13:26:37.167 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-07-29T13:26:37.169 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-07-29T13:26:37.169 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-07-29T13:26:37.298 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-07-29T13:26:37.337 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:37.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:37.439+0000 7f6766611500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-07-29T13:26:37.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:37.440+0000 7f6766611500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-07-29T13:26:37.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:37.440+0000 7f6766611500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-29T13:26:37.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:37.440+0000 7f6766611500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-29T13:26:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:37.440+0000 7f6766611500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-29T13:26:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:37.441+0000 7f6766611500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-07-29T13:26:40.289 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-07-29T13:26:40.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:40.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:40.418+0000 7f70eb411500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-07-29T13:26:40.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:40.418+0000 7f70eb411500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-07-29T13:26:40.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:40.418+0000 7f70eb411500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-29T13:26:40.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:40.418+0000 7f70eb411500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-29T13:26:40.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:40.418+0000 7f70eb411500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-29T13:26:40.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:40.419+0000 7f70eb411500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-07-29T13:26:43.267 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-07-29T13:26:43.303 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:43.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:43.398+0000 7f5e64615500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-07-29T13:26:43.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:43.398+0000 7f5e64615500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-07-29T13:26:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:43.398+0000 7f5e64615500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-29T13:26:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:43.398+0000 7f5e64615500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-29T13:26:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:43.398+0000 7f5e64615500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-29T13:26:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:43.399+0000 7f5e64615500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-07-29T13:26:46.252 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-07-29T13:26:46.288 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:46.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:46.382+0000 7f3280399500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-07-29T13:26:46.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:46.382+0000 7f3280399500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-07-29T13:26:46.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:46.382+0000 7f3280399500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-29T13:26:46.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:46.382+0000 7f3280399500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-29T13:26:46.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:46.382+0000 7f3280399500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-29T13:26:46.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:26:46.383+0000 7f3280399500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-07-29T13:26:49.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-07-29T13:26:49.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-07-29T13:26:49.310 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-29T13:26:49.311 INFO:tasks.ceph:role: osd.4 2023-07-29T13:26:49.311 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com 2023-07-29T13:26:49.311 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-29T13:26:49.383 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-29T13:26:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-29T13:26:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-29T13:26:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-07-29T13:26:49.384 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-29T13:26:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-07-29T13:26:49.384 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-29T13:26:49.385 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-29T13:26:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-29T13:26:49.385 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-29T13:26:49.407 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-07-29T13:26:49.409 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-07-29T13:26:49.410 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-07-29T13:26:49.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-07-29T13:26:49.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-07-29T13:26:49.634 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-29T13:26:49.634 INFO:tasks.ceph:role: osd.5 2023-07-29T13:26:49.634 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com 2023-07-29T13:26:49.635 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-29T13:26:49.704 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-29T13:26:49.704 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-29T13:26:49.705 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-29T13:26:49.705 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-07-29T13:26:49.705 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-29T13:26:49.705 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-07-29T13:26:49.705 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-29T13:26:49.705 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-29T13:26:49.705 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-29T13:26:49.705 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-29T13:26:49.728 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-07-29T13:26:49.729 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-07-29T13:26:49.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-07-29T13:26:49.804 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-07-29T13:26:49.879 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-07-29T13:26:49.949 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-29T13:26:49.950 INFO:tasks.ceph:role: osd.6 2023-07-29T13:26:49.950 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com 2023-07-29T13:26:49.951 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-29T13:26:50.019 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-29T13:26:50.019 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-29T13:26:50.020 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-29T13:26:50.020 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-07-29T13:26:50.020 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-29T13:26:50.020 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-07-29T13:26:50.020 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-29T13:26:50.020 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-29T13:26:50.021 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-29T13:26:50.021 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-29T13:26:50.043 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-07-29T13:26:50.044 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-07-29T13:26:50.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-07-29T13:26:50.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-07-29T13:26:50.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-07-29T13:26:50.264 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-29T13:26:50.264 INFO:tasks.ceph:role: osd.7 2023-07-29T13:26:50.265 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com 2023-07-29T13:26:50.265 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-29T13:26:50.333 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-29T13:26:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-29T13:26:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-29T13:26:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-07-29T13:26:50.334 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-29T13:26:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-07-29T13:26:50.335 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-29T13:26:50.335 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-29T13:26:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-29T13:26:50.335 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-29T13:26:50.357 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-07-29T13:26:50.359 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-07-29T13:26:50.359 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-07-29T13:26:50.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-07-29T13:26:50.505 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:50.602+0000 7fea9fbe0500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-07-29T13:26:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:50.602+0000 7fea9fbe0500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-07-29T13:26:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:50.602+0000 7fea9fbe0500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-29T13:26:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:50.602+0000 7fea9fbe0500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-29T13:26:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:50.602+0000 7fea9fbe0500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-29T13:26:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:50.602+0000 7fea9fbe0500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-07-29T13:26:53.448 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-07-29T13:26:53.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:53.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:53.577+0000 7f5217013500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-07-29T13:26:53.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:53.577+0000 7f5217013500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-07-29T13:26:53.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:53.577+0000 7f5217013500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-29T13:26:53.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:53.577+0000 7f5217013500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-29T13:26:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:53.577+0000 7f5217013500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-29T13:26:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:53.578+0000 7f5217013500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-07-29T13:26:56.416 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-07-29T13:26:56.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:56.545+0000 7f7559613500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-07-29T13:26:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:56.545+0000 7f7559613500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-07-29T13:26:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:56.545+0000 7f7559613500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-29T13:26:56.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:56.545+0000 7f7559613500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-29T13:26:56.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:56.545+0000 7f7559613500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-29T13:26:56.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:56.545+0000 7f7559613500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-07-29T13:26:59.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-07-29T13:26:59.434 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:26:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:59.528+0000 7f0dabe96500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-07-29T13:26:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:59.528+0000 7f0dabe96500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-07-29T13:26:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:59.528+0000 7f0dabe96500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-29T13:26:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:59.528+0000 7f0dabe96500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-29T13:26:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:59.528+0000 7f0dabe96500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-29T13:26:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:26:59.529+0000 7f0dabe96500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-07-29T13:27:02.372 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-07-29T13:27:02.408 INFO:tasks.ceph:Reading keys from all nodes... 2023-07-29T13:27:02.408 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:27:02.409 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-07-29T13:27:02.445 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:27:02.445 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-07-29T13:27:02.520 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:27:02.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-07-29T13:27:02.593 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:27:02.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-07-29T13:27:02.666 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:27:02.666 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-07-29T13:27:02.738 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:27:02.739 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-07-29T13:27:02.774 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:27:02.774 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-07-29T13:27:02.848 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:27:02.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-07-29T13:27:02.922 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:27:02.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-07-29T13:27:02.994 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:27:02.994 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-07-29T13:27:03.065 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:27:03.066 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-07-29T13:27:03.082 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:27:03.082 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-07-29T13:27:03.120 INFO:tasks.ceph:Adding keys to all mons... 2023-07-29T13:27:03.121 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-29T13:27:03.123 DEBUG:teuthology.orchestra.run.smithi045:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-29T13:27:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2023-07-29T13:27:03.164 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCKE8VkkMZrAhAA9V8FKoYeZ9Q+ojdYTjPLHw== 2023-07-29T13:27:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2023-07-29T13:27:03.165 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCNE8VkhQRLGhAAcqfojAE5VLcp5blsmN81IA== 2023-07-29T13:27:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2023-07-29T13:27:03.165 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCQE8VkEWsEGRAAdjeyuKQHnU3SueRjSmD2ow== 2023-07-29T13:27:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2023-07-29T13:27:03.165 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCTE8VkC2zOFxAA9o73BN9uMIjX/iUtFSt+HA== 2023-07-29T13:27:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2023-07-29T13:27:03.165 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCWE8VkbTzeFhAAmdzAGDmNauDpltRyFd8eww== 2023-07-29T13:27:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2023-07-29T13:27:03.166 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCKE8VkNLyrBxAAgKgp1hfW/lFO5+rBM+3iDQ== 2023-07-29T13:27:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2023-07-29T13:27:03.166 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCaE8VkY4oBJBAAcTPABdYT2r58OeBAF4NjPA== 2023-07-29T13:27:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2023-07-29T13:27:03.166 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCdE8VkOmGIIhAA1SVckI7TDJxmns54avZtvg== 2023-07-29T13:27:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2023-07-29T13:27:03.166 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCgE8VkTsWXIBAAeAXMcBB+TkNU+P/tY7wsUQ== 2023-07-29T13:27:03.167 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2023-07-29T13:27:03.167 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCjE8VkBnuKHxAA/ifM5eysofPvhRIZS8brIQ== 2023-07-29T13:27:03.167 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-07-29T13:27:03.167 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCKE8VkvZGiChAAjBYnsBrSh5xQNS05SvNlqg== 2023-07-29T13:27:03.167 INFO:teuthology.orchestra.run.smithi040.stdout:[client.1] 2023-07-29T13:27:03.168 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCKE8Vku/WbDhAAFBj8fgJho6wG+AeoddlOEg== 2023-07-29T13:27:03.190 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.y] 2023-07-29T13:27:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCKE8VkkMZrAhAA9V8FKoYeZ9Q+ojdYTjPLHw== 2023-07-29T13:27:03.191 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.0] 2023-07-29T13:27:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCNE8VkhQRLGhAAcqfojAE5VLcp5blsmN81IA== 2023-07-29T13:27:03.191 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.1] 2023-07-29T13:27:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCQE8VkEWsEGRAAdjeyuKQHnU3SueRjSmD2ow== 2023-07-29T13:27:03.192 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.2] 2023-07-29T13:27:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCTE8VkC2zOFxAA9o73BN9uMIjX/iUtFSt+HA== 2023-07-29T13:27:03.192 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.3] 2023-07-29T13:27:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCWE8VkbTzeFhAAmdzAGDmNauDpltRyFd8eww== 2023-07-29T13:27:03.192 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.x] 2023-07-29T13:27:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCKE8VkNLyrBxAAgKgp1hfW/lFO5+rBM+3iDQ== 2023-07-29T13:27:03.193 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.4] 2023-07-29T13:27:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaE8VkY4oBJBAAcTPABdYT2r58OeBAF4NjPA== 2023-07-29T13:27:03.193 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.5] 2023-07-29T13:27:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCdE8VkOmGIIhAA1SVckI7TDJxmns54avZtvg== 2023-07-29T13:27:03.193 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.6] 2023-07-29T13:27:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCgE8VkTsWXIBAAeAXMcBB+TkNU+P/tY7wsUQ== 2023-07-29T13:27:03.194 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.7] 2023-07-29T13:27:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCjE8VkBnuKHxAA/ifM5eysofPvhRIZS8brIQ== 2023-07-29T13:27:03.194 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2023-07-29T13:27:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCKE8VkvZGiChAAjBYnsBrSh5xQNS05SvNlqg== 2023-07-29T13:27:03.194 INFO:teuthology.orchestra.run.smithi045.stdout:[client.1] 2023-07-29T13:27:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCKE8Vku/WbDhAAFBj8fgJho6wG+AeoddlOEg== 2023-07-29T13:27:03.195 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-29T13:27:03.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-29T13:27:03.292 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.351 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.478 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.534 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-29T13:27:03.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-29T13:27:03.592 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.649 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.707 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.710 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.765 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.767 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-29T13:27:03.823 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-29T13:27:03.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-29T13:27:03.884 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-29T13:27:03.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-29T13:27:03.945 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-07-29T13:27:03.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-07-29T13:27:03.977 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-29T13:27:04.080 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-07-29T13:27:04.111 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-07-29T13:27:04.182 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-29T13:27:04.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-07-29T13:27:04.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-07-29T13:27:04.344 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-29T13:27:04.445 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-07-29T13:27:04.476 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:27:04.478 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-29T13:27:04.531 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-07-29T13:27:04.532 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-07-29T13:27:04.532 INFO:tasks.ceph.mon.a:Restarting daemon 2023-07-29T13:27:04.533 DEBUG:teuthology.orchestra.run.smithi040:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2023-07-29T13:27:04.535 INFO:tasks.ceph.mon.a:Started 2023-07-29T13:27:04.536 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-07-29T13:27:04.536 INFO:tasks.ceph.mon.c:Restarting daemon 2023-07-29T13:27:04.536 DEBUG:teuthology.orchestra.run.smithi040:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-07-29T13:27:04.538 INFO:tasks.ceph.mon.c:Started 2023-07-29T13:27:04.538 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-07-29T13:27:04.538 INFO:tasks.ceph.mon.b:Restarting daemon 2023-07-29T13:27:04.539 DEBUG:teuthology.orchestra.run.smithi045:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-07-29T13:27:04.573 INFO:tasks.ceph.mon.b:Started 2023-07-29T13:27:04.574 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-07-29T13:27:04.574 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-07-29T13:27:04.575 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2023-07-29T13:27:04.577 INFO:tasks.ceph.mgr.y:Started 2023-07-29T13:27:04.577 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-07-29T13:27:04.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2023-07-29T13:27:04.579 INFO:tasks.ceph.mgr.x:Started 2023-07-29T13:27:04.579 INFO:tasks.ceph:Setting crush tunables to default 2023-07-29T13:27:04.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2023-07-29T13:27:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.684+0000 7f58b58e3640 10 monclient: get_monmap_and_config 2023-07-29T13:27:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.684+0000 7f58b58e3640 10 monclient: build_initial_monmap 2023-07-29T13:27:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.684+0000 7f58b58e3640 10 monclient: monmap: 2023-07-29T13:27:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:27:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:27:04.686114+0000 2023-07-29T13:27:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:27:04.686114+0000 2023-07-29T13:27:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:04.688 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.685+0000 7f58b58e3640 1 Processor -- start 2023-07-29T13:27:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.685+0000 7f58b58e3640 1 -- start start 2023-07-29T13:27:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.685+0000 7f58b58e3640 10 monclient: init 2023-07-29T13:27:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2023-07-29T13:27:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f58b013dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 10 monclient(hunting): picked mon.noname-b con 0x7f58b00b3ae0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f58b00a8910 addr v1:172.21.15.45:6789/0 2023-07-29T13:27:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f58b0141320 addr v1:172.21.15.40:6789/0 2023-07-29T13:27:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b00b3eb0 con 0x7f58b0141320 2023-07-29T13:27:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b00a9950 con 0x7f58b00a8910 2023-07-29T13:27:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b00b0f10 con 0x7f58b00b3ae0 2023-07-29T13:27:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58b58e3640 10 monclient(hunting): authenticate will time out at 1346.819318s 2023-07-29T13:27:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58aeffd640 1 -- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f58b013dc90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3301/0 2023-07-29T13:27:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58af7fe640 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f58b0141320 legacy=0x7f58b014b520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2023-07-29T13:27:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58aeffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f58b013dc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-29T13:27:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.686+0000 7f58ae7fc640 1 -- >> v1:172.21.15.45:6789/0 conn(0x7f58b00a8910 legacy=0x7f58b0140c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6789/0 2023-07-29T13:27:04.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.887+0000 7f58af7fe640 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f58b0141320 legacy=0x7f58b014b520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2023-07-29T13:27:04.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.887+0000 7f58aeffd640 1 -- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f58b013dc90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3301/0 2023-07-29T13:27:04.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.887+0000 7f58aeffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f58b013dc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-29T13:27:04.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:04.887+0000 7f58ae7fc640 1 -- >> v1:172.21.15.45:6789/0 conn(0x7f58b00a8910 legacy=0x7f58b0140c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6789/0 2023-07-29T13:27:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:05.288+0000 7f58af7fe640 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f58b0141320 legacy=0x7f58b014b520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2023-07-29T13:27:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:05.288+0000 7f58aeffd640 1 -- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f58b013dc90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3301/0 2023-07-29T13:27:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:05.288+0000 7f58aeffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f58b013dc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-29T13:27:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:05.288+0000 7f58ae7fc640 1 -- >> v1:172.21.15.45:6789/0 conn(0x7f58b00a8910 legacy=0x7f58b0140c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6789/0 2023-07-29T13:27:06.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:06.089+0000 7f58aeffd640 1 -- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f58b013dc90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3301/0 2023-07-29T13:27:06.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:06.089+0000 7f58af7fe640 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f58b0141320 legacy=0x7f58b014b520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2023-07-29T13:27:06.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:06.089+0000 7f58aeffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f58b013dc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-29T13:27:06.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:06.089+0000 7f58ae7fc640 1 -- >> v1:172.21.15.45:6789/0 conn(0x7f58b00a8910 legacy=0x7f58b0140c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6789/0 2023-07-29T13:27:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 10 monclient(hunting): tick 2023-07-29T13:27:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 10 monclient(hunting): _check_auth_tickets 2023-07-29T13:27:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 1 monclient(hunting): continuing hunt 2023-07-29T13:27:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 10 monclient(hunting): _reopen_session rank -1 2023-07-29T13:27:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 1 -- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f58b013dc90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-29T13:27:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f58b013dc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 1 -- >> v1:172.21.15.45:6789/0 conn(0x7f58b00a8910 legacy=0x7f58b0140c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-29T13:27:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f58b0141320 legacy=0x7f58b014b520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-29T13:27:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 10 monclient: _add_conns ranks=[3,4] 2023-07-29T13:27:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c0036d0 0x7f589c005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 10 monclient(hunting): picked mon.noname-a con 0x7f589c0036d0 addr v2:172.21.15.40:3300/0 2023-07-29T13:27:07.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c006190 0x7f589c008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:07.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 10 monclient(hunting): picked mon.noname-c con 0x7f589c006190 addr v2:172.21.15.45:3300/0 2023-07-29T13:27:07.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:07.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589c008cd0 con 0x7f589c0036d0 2023-07-29T13:27:07.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:07.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589c008e40 con 0x7f589c006190 2023-07-29T13:27:07.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58acff9640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:07.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58aeffd640 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f589c0036d0 msgr2=0x7f589c005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2023-07-29T13:27:07.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58aeffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c0036d0 0x7f589c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-29T13:27:07.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58ae7fc640 1 -- >> v2:172.21.15.45:3300/0 conn(0x7f589c006190 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:3300/0 2023-07-29T13:27:07.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.686+0000 7f58ae7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c006190 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-29T13:27:07.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.887+0000 7f58aeffd640 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f589c0036d0 msgr2=0x7f589c005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2023-07-29T13:27:07.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.887+0000 7f58aeffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c0036d0 0x7f589c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-29T13:27:07.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.887+0000 7f58ae7fc640 1 -- >> v2:172.21.15.45:3300/0 conn(0x7f589c006190 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:3300/0 2023-07-29T13:27:07.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:07.887+0000 7f58ae7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c006190 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-29T13:27:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:08.288+0000 7f58aeffd640 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f589c0036d0 msgr2=0x7f589c005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2023-07-29T13:27:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:08.288+0000 7f58aeffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c0036d0 0x7f589c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-29T13:27:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:08.288+0000 7f58ae7fc640 1 -- >> v2:172.21.15.45:3300/0 conn(0x7f589c006190 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:3300/0 2023-07-29T13:27:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:08.288+0000 7f58ae7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c006190 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-29T13:27:09.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:09.089+0000 7f58aeffd640 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f589c0036d0 msgr2=0x7f589c005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2023-07-29T13:27:09.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:09.089+0000 7f58aeffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c0036d0 0x7f589c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-29T13:27:09.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:09.089+0000 7f58ae7fc640 1 -- >> v2:172.21.15.45:3300/0 conn(0x7f589c006190 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:3300/0 2023-07-29T13:27:09.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:09.089+0000 7f58ae7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c006190 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-29T13:27:10.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 10 monclient(hunting): tick 2023-07-29T13:27:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 10 monclient(hunting): _check_auth_tickets 2023-07-29T13:27:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 1 monclient(hunting): continuing hunt 2023-07-29T13:27:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 10 monclient(hunting): _reopen_session rank -1 2023-07-29T13:27:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 1 -- >> v2:172.21.15.45:3300/0 conn(0x7f589c006190 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-29T13:27:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c006190 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f589c0036d0 msgr2=0x7f589c005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-29T13:27:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c0036d0 0x7f589c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2023-07-29T13:27:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58af7fe640 1 -- reap_dead start 2023-07-29T13:27:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c0090d0 0x7f589c0094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 10 monclient(hunting): picked mon.noname-c con 0x7f589c0090d0 addr v2:172.21.15.45:3300/0 2023-07-29T13:27:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 10 monclient(hunting): picked mon.noname-b con 0x7f58b00b3ae0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f589c003b30 addr v1:172.21.15.40:6789/0 2023-07-29T13:27:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b00a9c30 con 0x7f589c003b30 2023-07-29T13:27:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589c008cd0 con 0x7f589c0090d0 2023-07-29T13:27:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58ae7fc640 1 -- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3301/0 2023-07-29T13:27:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589c008e40 con 0x7f58b00b3ae0 2023-07-29T13:27:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58acff9640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58ae7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-29T13:27:10.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58aeffd640 1 -- >> v2:172.21.15.45:3300/0 conn(0x7f589c0090d0 msgr2=0x7f589c0094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:3300/0 2023-07-29T13:27:10.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58af7fe640 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f589c003b30 legacy=0x7f589c00ba90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2023-07-29T13:27:10.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.686+0000 7f58aeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c0090d0 0x7f589c0094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-29T13:27:10.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.887+0000 7f58ae7fc640 1 -- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3301/0 2023-07-29T13:27:10.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.887+0000 7f58af7fe640 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f589c003b30 legacy=0x7f589c00ba90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2023-07-29T13:27:10.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.887+0000 7f58ae7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-29T13:27:10.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.887+0000 7f58aeffd640 1 -- >> v2:172.21.15.45:3300/0 conn(0x7f589c0090d0 msgr2=0x7f589c0094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:3300/0 2023-07-29T13:27:10.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:10.887+0000 7f58aeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c0090d0 0x7f589c0094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-29T13:27:11.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:11.288+0000 7f58ae7fc640 1 -- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3301/0 2023-07-29T13:27:11.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:11.288+0000 7f58af7fe640 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f589c003b30 legacy=0x7f589c00ba90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2023-07-29T13:27:11.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:11.288+0000 7f58ae7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-29T13:27:11.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:11.288+0000 7f58aeffd640 1 -- >> v2:172.21.15.45:3300/0 conn(0x7f589c0090d0 msgr2=0x7f589c0094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:3300/0 2023-07-29T13:27:11.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:11.288+0000 7f58aeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c0090d0 0x7f589c0094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-29T13:27:12.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:12.089+0000 7f58af7fe640 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f589c003b30 legacy=0x7f589c00ba90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2023-07-29T13:27:12.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:12.089+0000 7f58ae7fc640 1 -- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3301/0 2023-07-29T13:27:12.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:12.089+0000 7f58ae7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-29T13:27:12.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:12.089+0000 7f58aeffd640 1 -- >> v2:172.21.15.45:3300/0 conn(0x7f589c0090d0 msgr2=0x7f589c0094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:3300/0 2023-07-29T13:27:12.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:12.089+0000 7f58aeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c0090d0 0x7f589c0094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-29T13:27:12.814 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:00:08.148 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:27:12.817 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:00:08.151 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:27:12.838 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:00:08.122 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:27:13.149 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:00:08.484 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:27:13.151 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:00:08.486 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:27:13.173 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:00:08.457 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:27:13.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 10 monclient(hunting): tick 2023-07-29T13:27:13.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 10 monclient(hunting): _check_auth_tickets 2023-07-29T13:27:13.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 1 monclient(hunting): continuing hunt 2023-07-29T13:27:13.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 10 monclient(hunting): _reopen_session rank -1 2023-07-29T13:27:13.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 1 -- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-29T13:27:13.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:13.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 1 -- >> v2:172.21.15.45:3300/0 conn(0x7f589c0090d0 msgr2=0x7f589c0094a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-29T13:27:13.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589c0090d0 0x7f589c0094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:13.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f589c003b30 legacy=0x7f589c00ba90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-29T13:27:13.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 10 monclient: _add_conns ranks=[1,3] 2023-07-29T13:27:13.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f589c00d2a0 addr v1:172.21.15.45:6789/0 2023-07-29T13:27:13.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 0x7f589c013390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:13.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 10 monclient(hunting): picked mon.noname-a con 0x7f589c00de80 addr v2:172.21.15.40:3300/0 2023-07-29T13:27:13.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b00b3eb0 con 0x7f589c00d2a0 2023-07-29T13:27:13.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:13.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589c008cd0 con 0x7f589c00de80 2023-07-29T13:27:13.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58acff9640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:13.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58ae7fc640 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 msgr2=0x7f589c013390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2023-07-29T13:27:13.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58ae7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 0x7f589c013390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-29T13:27:13.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.686+0000 7f58aeffd640 1 -- >> v1:172.21.15.45:6789/0 conn(0x7f589c00d2a0 legacy=0x7f589c00d6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6789/0 2023-07-29T13:27:13.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.887+0000 7f58ae7fc640 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 msgr2=0x7f589c013390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2023-07-29T13:27:13.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.887+0000 7f58ae7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 0x7f589c013390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-29T13:27:13.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:13.887+0000 7f58aeffd640 1 -- >> v1:172.21.15.45:6789/0 conn(0x7f589c00d2a0 legacy=0x7f589c00d6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6789/0 2023-07-29T13:27:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:14.288+0000 7f58ae7fc640 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 msgr2=0x7f589c013390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2023-07-29T13:27:14.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:14.288+0000 7f58ae7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 0x7f589c013390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-29T13:27:14.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:14.288+0000 7f58aeffd640 1 -- >> v1:172.21.15.45:6789/0 conn(0x7f589c00d2a0 legacy=0x7f589c00d6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6789/0 2023-07-29T13:27:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:15.259+0000 7f58ae7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 0x7f589c013390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:15.294+0000 7f58ae7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 0x7f589c013390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59448/0 (socket says 172.21.15.40:59448) 2023-07-29T13:27:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:15.294+0000 7f58ae7fc640 1 -- 172.21.15.40:0/1637707874 learned_addr learned my addr 172.21.15.40:0/1637707874 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:27:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:15.294+0000 7f58ae7fc640 10 monclient(hunting): get_auth_request con 0x7f589c00de80 auth_method 0 2023-07-29T13:27:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:15.294+0000 7f58ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:15.294+0000 7f58ae7fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:15.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:15.294+0000 7f58ae7fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:15.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:15.391+0000 7f58ae7fc640 1 -- 172.21.15.40:0/1637707874 >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 msgr2=0x7f589c013390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-29T13:27:15.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:15.391+0000 7f58ae7fc640 1 -- 172.21.15.40:0/1637707874 >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 msgr2=0x7f589c013390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-29T13:27:15.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:15.391+0000 7f58ae7fc640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 0x7f589c013390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-29T13:27:15.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:15.391+0000 7f58ae7fc640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 0x7f589c013390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-29T13:27:16.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 10 monclient(hunting): tick 2023-07-29T13:27:16.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 10 monclient(hunting): _check_auth_tickets 2023-07-29T13:27:16.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 1 monclient(hunting): continuing hunt 2023-07-29T13:27:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 10 monclient(hunting): _reopen_session rank -1 2023-07-29T13:27:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 1 -- 172.21.15.40:0/1637707874 >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 msgr2=0x7f589c013390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-29T13:27:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.40:3300/0 conn(0x7f589c00de80 0x7f589c013390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 1 -- 172.21.15.40:0/1637707874 >> v1:172.21.15.45:6789/0 conn(0x7f589c00d2a0 legacy=0x7f589c00d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2023-07-29T13:27:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58af7fe640 1 -- 172.21.15.40:0/1637707874 reap_dead start 2023-07-29T13:27:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 10 monclient(hunting): picked mon.noname-b con 0x7f58b00b3ae0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 10 monclient(hunting): picked mon.noname-c con 0x7f589c0094a0 addr v2:172.21.15.45:3300/0 2023-07-29T13:27:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.686+0000 7f58acff9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f589c013990 addr v1:172.21.15.45:6789/0 2023-07-29T13:27:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.687+0000 7f58acff9640 1 -- 172.21.15.40:0/1637707874 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b00a9c30 con 0x7f589c013990 2023-07-29T13:27:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.687+0000 7f58acff9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.687+0000 7f58acff9640 1 -- 172.21.15.40:0/1637707874 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589c008cd0 con 0x7f589c0094a0 2023-07-29T13:27:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.687+0000 7f58acff9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.687+0000 7f58acff9640 1 -- 172.21.15.40:0/1637707874 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589c008e40 con 0x7f58b00b3ae0 2023-07-29T13:27:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.687+0000 7f58acff9640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.687+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.687+0000 7f58aeffd640 10 monclient(hunting): get_auth_request con 0x7f58b00b3ae0 auth_method 0 2023-07-29T13:27:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.687+0000 7f58aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.687+0000 7f58aeffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.687+0000 7f58aeffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.697+0000 7f58aeffd640 1 -- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-29T13:27:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.697+0000 7f58aeffd640 1 -- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-29T13:27:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.697+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-29T13:27:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.697+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-29T13:27:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.707+0000 7f58ae7fc640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.707+0000 7f58ae7fc640 10 monclient(hunting): get_auth_request con 0x7f589c0094a0 auth_method 0 2023-07-29T13:27:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.707+0000 7f58ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.707+0000 7f58ae7fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.707+0000 7f58ae7fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:16.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.709+0000 7f58ae7fc640 1 -- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 msgr2=0x7f589c00d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-07-29T13:27:16.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.709+0000 7f58ae7fc640 1 -- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 msgr2=0x7f589c00d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-29T13:27:16.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.709+0000 7f58ae7fc640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-29T13:27:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.709+0000 7f58ae7fc640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-29T13:27:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.898+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.898+0000 7f58aeffd640 10 monclient(hunting): get_auth_request con 0x7f58b00b3ae0 auth_method 0 2023-07-29T13:27:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.898+0000 7f58aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.898+0000 7f58aeffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.898+0000 7f58aeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2023-07-29T13:27:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.899+0000 7f58aeffd640 1 -- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-29T13:27:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.899+0000 7f58aeffd640 1 -- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-29T13:27:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.899+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-29T13:27:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.899+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-29T13:27:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.933+0000 7f58ae7fc640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.940+0000 7f58ae7fc640 10 monclient(hunting): get_auth_request con 0x7f589c0094a0 auth_method 0 2023-07-29T13:27:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.940+0000 7f58ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.940+0000 7f58ae7fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:16.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.940+0000 7f58ae7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2023-07-29T13:27:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.978+0000 7f58ae7fc640 1 -- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 msgr2=0x7f589c00d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-29T13:27:16.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.978+0000 7f58ae7fc640 1 -- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 msgr2=0x7f589c00d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-29T13:27:16.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.978+0000 7f58ae7fc640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-29T13:27:16.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:16.978+0000 7f58ae7fc640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-29T13:27:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.300+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.300+0000 7f58aeffd640 10 monclient(hunting): get_auth_request con 0x7f58b00b3ae0 auth_method 0 2023-07-29T13:27:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.300+0000 7f58aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.300+0000 7f58aeffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.300+0000 7f58aeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2023-07-29T13:27:17.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.301+0000 7f58aeffd640 1 -- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-29T13:27:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.301+0000 7f58aeffd640 1 -- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-29T13:27:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.301+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-29T13:27:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.301+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-29T13:27:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.379+0000 7f58ae7fc640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.379+0000 7f58ae7fc640 10 monclient(hunting): get_auth_request con 0x7f589c0094a0 auth_method 0 2023-07-29T13:27:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.379+0000 7f58ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.379+0000 7f58ae7fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.379+0000 7f58ae7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2023-07-29T13:27:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.380+0000 7f58ae7fc640 1 -- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 msgr2=0x7f589c00d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-29T13:27:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.380+0000 7f58ae7fc640 1 -- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 msgr2=0x7f589c00d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-29T13:27:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.380+0000 7f58ae7fc640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-29T13:27:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.380+0000 7f58ae7fc640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-29T13:27:17.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.992+0000 7f58adffb640 1 -- 172.21.15.40:0/1637707874 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458262940 0 0) 0x7f58b00a9c30 con 0x7f589c013990 2023-07-29T13:27:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.992+0000 7f58adffb640 10 monclient(hunting): _negotiate 2023-07-29T13:27:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.992+0000 7f58adffb640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.992+0000 7f58adffb640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.992+0000 7f58adffb640 10 monclient(hunting): my global_id is 4102 2023-07-29T13:27:17.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:17.992+0000 7f58adffb640 1 -- 172.21.15.40:0/1637707874 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5898003520 con 0x7f589c013990 2023-07-29T13:27:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.113+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.114+0000 7f58aeffd640 10 monclient(hunting): get_auth_request con 0x7f58b00b3ae0 auth_method 0 2023-07-29T13:27:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.114+0000 7f58aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.114+0000 7f58aeffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.114+0000 7f58aeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2023-07-29T13:27:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.116+0000 7f58aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.116+0000 7f58aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.116+0000 7f58aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:18.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.120+0000 7f58aeffd640 10 monclient(hunting): handle_auth_done global_id 4100 payload 274 2023-07-29T13:27:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.120+0000 7f58aeffd640 1 -- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 msgr2=0x7f589c00d670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-29T13:27:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.120+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.120+0000 7f58aeffd640 1 -- 172.21.15.40:0/1637707874 >> v1:172.21.15.45:6789/0 conn(0x7f589c013990 legacy=0x7f589c013d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.120+0000 7f58aeffd640 10 monclient: _finish_hunting 0 2023-07-29T13:27:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.120+0000 7f58aeffd640 1 monclient: found mon.noname-b 2023-07-29T13:27:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.120+0000 7f58aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:18.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.120+0000 7f58aeffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:27:18.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.120+0000 7f58aeffd640 1 -- 172.21.15.40:0/1637707874 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b00a9950 con 0x7f58b00b3ae0 2023-07-29T13:27:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.125+0000 7f58aeffd640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f58a0006fd0 tx=0x7f58a002bf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dede1c1a949965e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:18.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 1 -- 172.21.15.40:0/1637707874 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0003d50 con 0x7f58b00b3ae0 2023-07-29T13:27:18.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:27:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 10 monclient: dump: 2023-07-29T13:27:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:27:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:18.200 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 10 monclient: _finish_auth 0 2023-07-29T13:27:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 10 monclient: _check_auth_tickets 2023-07-29T13:27:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58b58e3640 5 monclient: authenticate success, global_id 4100 2023-07-29T13:27:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 1 -- 172.21.15.40:0/1637707874 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a0030070 con 0x7f58b00b3ae0 2023-07-29T13:27:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 1 -- 172.21.15.40:0/1637707874 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a002cd60 con 0x7f58b00b3ae0 2023-07-29T13:27:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58b58e3640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58b58e3640 4 set_mon_vals no callback set 2023-07-29T13:27:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:27:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.194+0000 7f58adffb640 10 monclient: dump: 2023-07-29T13:27:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:27:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:18.206 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.195+0000 7f58b58e3640 10 monclient: shutdown 2023-07-29T13:27:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.195+0000 7f58b58e3640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.195+0000 7f58b58e3640 1 -- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 msgr2=0x7f589c008640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.195+0000 7f58b58e3640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f58a0006fd0 tx=0x7f58a002bf70 comp rx=0 tx=0).stop 2023-07-29T13:27:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.195+0000 7f58b58e3640 1 -- 172.21.15.40:0/1637707874 shutdown_connections 2023-07-29T13:27:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.195+0000 7f58b58e3640 1 --2- 172.21.15.40:0/1637707874 >> v2:172.21.15.45:3300/0 conn(0x7f589c0094a0 0x7f589c00d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.195+0000 7f58b58e3640 1 --2- 172.21.15.40:0/1637707874 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f58b00b3ae0 0x7f589c008640 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.195+0000 7f58b58e3640 1 -- 172.21.15.40:0/1637707874 >> 172.21.15.40:0/1637707874 conn(0x7f58b0070fc0 msgr2=0x7f58b00713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.195+0000 7f58b58e3640 1 -- 172.21.15.40:0/1637707874 shutdown_connections 2023-07-29T13:27:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.195+0000 7f58b58e3640 1 -- 172.21.15.40:0/1637707874 wait complete. 2023-07-29T13:27:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.196+0000 7f58b58e3640 10 monclient: build_initial_monmap 2023-07-29T13:27:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.196+0000 7f58b58e3640 10 monclient: monmap: 2023-07-29T13:27:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:27:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:27:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:27:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:18.209 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:18.209 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:18.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.196+0000 7f58b58e3640 1 librados: starting msgr at 2023-07-29T13:27:18.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.196+0000 7f58b58e3640 1 librados: starting objecter 2023-07-29T13:27:18.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.196+0000 7f58b58e3640 1 Processor -- start 2023-07-29T13:27:18.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.196+0000 7f58b58e3640 1 -- start start 2023-07-29T13:27:18.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.196+0000 7f58b58e3640 1 librados: setting wanted keys 2023-07-29T13:27:18.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.196+0000 7f58b58e3640 1 librados: calling monclient init 2023-07-29T13:27:18.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.196+0000 7f58b58e3640 10 monclient: init 2023-07-29T13:27:18.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:18.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T13:27:18.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f589c0094a0 0x7f58b013b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:18.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 10 monclient(hunting): picked mon.noname-a con 0x7f589c0094a0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:18.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f589c013990 0x7f58b0144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 10 monclient(hunting): picked mon.noname-c con 0x7f589c013990 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b00b3ae0 0x7f58b01d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 10 monclient(hunting): picked mon.noname-b con 0x7f58b00b3ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b0144570 con 0x7f589c0094a0 2023-07-29T13:27:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b01450d0 con 0x7f58b00b3ae0 2023-07-29T13:27:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b0144700 con 0x7f589c013990 2023-07-29T13:27:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.197+0000 7f58b58e3640 10 monclient(hunting): authenticate will time out at 1360.330488s 2023-07-29T13:27:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.198+0000 7f58af7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b00b3ae0 0x7f58b01d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.198+0000 7f58af7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b00b3ae0 0x7f58b01d8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:33430/0 (socket says 172.21.15.40:33430) 2023-07-29T13:27:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.198+0000 7f58af7fe640 1 -- 172.21.15.40:0/3269615778 learned_addr learned my addr 172.21.15.40:0/3269615778 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:27:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.198+0000 7f58af7fe640 10 monclient(hunting): get_auth_request con 0x7f58b00b3ae0 auth_method 0 2023-07-29T13:27:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.198+0000 7f58af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.198+0000 7f58af7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.198+0000 7f58af7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.199+0000 7f58af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.199+0000 7f58af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.199+0000 7f58af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.211+0000 7f58af7fe640 10 monclient(hunting): handle_auth_done global_id 4105 payload 824 2023-07-29T13:27:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.211+0000 7f58af7fe640 1 -- 172.21.15.40:0/3269615778 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f589c013990 msgr2=0x7f58b0144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.211+0000 7f58af7fe640 1 --2- 172.21.15.40:0/3269615778 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f589c013990 0x7f58b0144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.211+0000 7f58af7fe640 1 -- 172.21.15.40:0/3269615778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f589c0094a0 msgr2=0x7f58b013b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.211+0000 7f58af7fe640 1 --2- 172.21.15.40:0/3269615778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f589c0094a0 0x7f58b013b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.211+0000 7f58af7fe640 10 monclient: _finish_hunting 0 2023-07-29T13:27:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.211+0000 7f58af7fe640 1 monclient: found mon.noname-b 2023-07-29T13:27:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.211+0000 7f58af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.211+0000 7f58af7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:27:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.211+0000 7f58af7fe640 1 -- 172.21.15.40:0/3269615778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b00b3eb0 con 0x7f58b00b3ae0 2023-07-29T13:27:18.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.214+0000 7f58af7fe640 1 --2- 172.21.15.40:0/3269615778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b00b3ae0 0x7f58b01d8af0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f589c006100 tx=0x7f58a800c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:18.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.214+0000 7f5897fff640 10 client.?.objecter ms_handle_connect 0x7f58b00b3ae0 2023-07-29T13:27:18.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.214+0000 7f5897fff640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:18.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.225+0000 7f5897fff640 1 -- 172.21.15.40:0/3269615778 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8013400 con 0x7f58b00b3ae0 2023-07-29T13:27:18.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.225+0000 7f5897fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:18.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.225+0000 7f5897fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:27:18.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.225+0000 7f5897fff640 10 monclient: dump: 2023-07-29T13:27:18.228 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:27:18.229 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:18.229 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:18.229 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:18.229 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:18.229 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:18.229 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:18.229 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:18.230 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:18.230 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:18.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.225+0000 7f5897fff640 10 monclient: _finish_auth 0 2023-07-29T13:27:18.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.225+0000 7f5897fff640 10 monclient: _check_auth_tickets 2023-07-29T13:27:18.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.225+0000 7f5897fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:18.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f5897fff640 1 -- 172.21.15.40:0/3269615778 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a8013580 con 0x7f58b00b3ae0 2023-07-29T13:27:18.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f58b58e3640 5 monclient: authenticate success, global_id 4105 2023-07-29T13:27:18.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f5897fff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:18.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f5897fff640 1 -- 172.21.15.40:0/3269615778 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8013930 con 0x7f58b00b3ae0 2023-07-29T13:27:18.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f58b58e3640 10 monclient: _renew_subs 2023-07-29T13:27:18.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f58b58e3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:18.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f58b58e3640 1 -- 172.21.15.40:0/3269615778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b00b6d20 con 0x7f58b00b3ae0 2023-07-29T13:27:18.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f58ad7fa640 4 set_mon_vals no callback set 2023-07-29T13:27:18.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f5897fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:18.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f5897fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:18.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f5897fff640 10 monclient: dump: 2023-07-29T13:27:18.232 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:27:18.232 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:18.232 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:18.232 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:18.233 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:18.233 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:18.233 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:18.233 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:18.233 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:18.233 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:18.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f58b58e3640 10 client.4105.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:18.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f58b58e3640 10 monclient: _renew_subs 2023-07-29T13:27:18.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f58b58e3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:18.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f58b58e3640 1 -- 172.21.15.40:0/3269615778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b00af660 con 0x7f58b00b3ae0 2023-07-29T13:27:18.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.226+0000 7f58b58e3640 1 librados: init done 2023-07-29T13:27:18.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.227+0000 7f58b58e3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:18.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.227+0000 7f58b58e3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:18.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.227+0000 7f58b58e3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:18.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.227+0000 7f58b58e3640 1 -- 172.21.15.40:0/3269615778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b0141d40 con 0x7f58b00b3ae0 2023-07-29T13:27:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.274+0000 7f5897fff640 1 -- 172.21.15.40:0/3269615778 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f58a801c3f0 con 0x7f58b00b3ae0 2023-07-29T13:27:18.357 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:18.355+0000 7ff142a2c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-29T13:27:18.427 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:18.426+0000 7f9c90e32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-29T13:27:18.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.487+0000 7f5897fff640 1 -- 172.21.15.40:0/3269615778 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f58a800e040 con 0x7f58b00b3ae0 2023-07-29T13:27:18.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.487+0000 7f5897fff640 10 client.4105.objecter ms_dispatch 0x7f58b0000b90 osd_map(1..1 src has 1..1) v4 2023-07-29T13:27:18.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.487+0000 7f5897fff640 3 client.4105.objecter handle_osd_map got epochs [1,1] > 0 2023-07-29T13:27:18.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.487+0000 7f5897fff640 3 client.4105.objecter handle_osd_map decoding full epoch 1 2023-07-29T13:27:18.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.487+0000 7f5897fff640 20 client.4105.objecter dump_active .. 0 homeless 2023-07-29T13:27:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.487+0000 7f5897fff640 1 -- 172.21.15.40:0/3269615778 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f58a80123f0 con 0x7f58b00b3ae0 2023-07-29T13:27:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.487+0000 7f5897fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.487+0000 7f5897fff640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.557+0000 7f58b58e3640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2023-07-29T13:27:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.557+0000 7f58b58e3640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2023-07-29T13:27:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.557+0000 7f58b58e3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:18.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.557+0000 7f58b58e3640 1 -- 172.21.15.40:0/3269615778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f58b00af910 con 0x7f58b00b3ae0 2023-07-29T13:27:18.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.832+0000 7f5897fff640 1 -- 172.21.15.40:0/3269615778 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f58a801c880 con 0x7f58b00b3ae0 2023-07-29T13:27:18.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.832+0000 7f5897fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2023-07-29T13:27:18.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.832+0000 7f5897fff640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2023-07-29T13:27:18.834 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2023-07-29T13:27:18.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 10 librados: watch_flush enter 2023-07-29T13:27:18.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 10 librados: watch_flush exit 2023-07-29T13:27:18.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 20 client.4105.objecter shutdown clearing up homeless session... 2023-07-29T13:27:18.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 10 client.4105.objecter successfully canceled tick 2023-07-29T13:27:18.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 10 monclient: shutdown 2023-07-29T13:27:18.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:18.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 1 -- 172.21.15.40:0/3269615778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b00b3ae0 msgr2=0x7f58b01d8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:18.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 1 --2- 172.21.15.40:0/3269615778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b00b3ae0 0x7f58b01d8af0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f589c006100 tx=0x7f58a800c7e0 comp rx=0 tx=0).stop 2023-07-29T13:27:18.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 1 -- 172.21.15.40:0/3269615778 shutdown_connections 2023-07-29T13:27:18.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 1 --2- 172.21.15.40:0/3269615778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b00b3ae0 0x7f58b01d8af0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:18.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 1 --2- 172.21.15.40:0/3269615778 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f589c013990 0x7f58b0144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:18.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 1 --2- 172.21.15.40:0/3269615778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f589c0094a0 0x7f58b013b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:18.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 1 -- 172.21.15.40:0/3269615778 >> 172.21.15.40:0/3269615778 conn(0x7f58b0070fc0 msgr2=0x7f58b013dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:18.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 1 -- 172.21.15.40:0/3269615778 shutdown_connections 2023-07-29T13:27:18.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 1 -- 172.21.15.40:0/3269615778 wait complete. 2023-07-29T13:27:18.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:18.833+0000 7f58b58e3640 1 librados: shutdown 2023-07-29T13:27:18.845 INFO:tasks.ceph:check_enable_crimson: False 2023-07-29T13:27:18.846 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-07-29T13:27:18.847 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:27:18.847 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-07-29T13:27:18.879 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:27:18.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-07-29T13:27:18.950 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:27:18.950 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-07-29T13:27:19.022 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:27:19.022 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-07-29T13:27:19.092 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:27:19.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-07-29T13:27:19.123 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:27:19.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-07-29T13:27:19.178 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:19.177+0000 7ff142a2c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-29T13:27:19.194 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:27:19.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-07-29T13:27:19.265 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-07-29T13:27:19.266 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-07-29T13:27:19.302 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:19.301+0000 7f9c90e32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-29T13:27:19.330 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:19.328+0000 7ff142a2c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-29T13:27:19.338 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new 6069a944-26b4-4b00-94e2-f91c485e8957 0 2023-07-29T13:27:19.390 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:19.389+0000 7ff142a2c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-29T13:27:19.438 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:19.436+0000 7ff142a2c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-29T13:27:19.447 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:19.446+0000 7f9c90e32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-29T13:27:19.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.481+0000 7f10b28bc640 10 monclient: get_monmap_and_config 2023-07-29T13:27:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.481+0000 7f10b28bc640 10 monclient: build_initial_monmap 2023-07-29T13:27:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.481+0000 7f10b28bc640 10 monclient: monmap: 2023-07-29T13:27:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:27:19.482178+0000 2023-07-29T13:27:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:27:19.482178+0000 2023-07-29T13:27:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:19.484 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 1 Processor -- start 2023-07-29T13:27:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 1 -- start start 2023-07-29T13:27:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 10 monclient: init 2023-07-29T13:27:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2023-07-29T13:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f10ac1420c0 addr v1:172.21.15.40:6789/0 2023-07-29T13:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f10ac142490 0x7f10ac06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 10 monclient(hunting): picked mon.noname-b con 0x7f10ac142490 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10ac06f620 0x7f10ac1491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 10 monclient(hunting): picked mon.noname-c con 0x7f10ac06f620 addr v2:172.21.15.45:3300/0 2023-07-29T13:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac071080 con 0x7f10ac1420c0 2023-07-29T13:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac142c90 con 0x7f10ac06f620 2023-07-29T13:27:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac1428c0 con 0x7f10ac142490 2023-07-29T13:27:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.482+0000 7f10b28bc640 10 monclient(hunting): authenticate will time out at 1362.710079s 2023-07-29T13:27:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.483+0000 7f10b0e59640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10ac06f620 0x7f10ac1491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.483+0000 7f10b0e59640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10ac06f620 0x7f10ac1491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38470/0 (socket says 172.21.15.45:38470) 2023-07-29T13:27:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.483+0000 7f10b0e59640 1 -- 172.21.15.45:0/8804708 learned_addr learned my addr 172.21.15.45:0/8804708 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-07-29T13:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.483+0000 7f10b0e59640 10 monclient(hunting): get_auth_request con 0x7f10ac06f620 auth_method 0 2023-07-29T13:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.483+0000 7f10b0e59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.483+0000 7f10b0e59640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.483+0000 7f10b0e59640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.484+0000 7f10b0e59640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.484+0000 7f10b0e59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.484+0000 7f10b0e59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.486+0000 7f10b0e59640 10 monclient(hunting): handle_auth_done global_id 4111 payload 274 2023-07-29T13:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.486+0000 7f10b0e59640 1 -- 172.21.15.45:0/8804708 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f10ac142490 msgr2=0x7f10ac06f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.486+0000 7f10b0e59640 1 --2- 172.21.15.45:0/8804708 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f10ac142490 0x7f10ac06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.486+0000 7f10b0e59640 1 -- 172.21.15.45:0/8804708 >> v1:172.21.15.40:6789/0 conn(0x7f10ac1420c0 legacy=0x7f10ac13bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.486+0000 7f10b0e59640 10 monclient: _finish_hunting 0 2023-07-29T13:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.486+0000 7f10b0e59640 1 monclient: found mon.noname-c 2023-07-29T13:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.486+0000 7f10b0e59640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.486+0000 7f10b0e59640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.486+0000 7f10b0e59640 1 -- 172.21.15.45:0/8804708 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac0712f0 con 0x7f10ac06f620 2023-07-29T13:27:19.500 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:19.498+0000 7f9c90e32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-29T13:27:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.520+0000 7f10b0e59640 1 --2- 172.21.15.45:0/8804708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ac06f620 0x7f10ac1491e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009e50 tx=0x7f10a402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=883d3423b361940d server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:19.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 1 -- 172.21.15.45:0/8804708 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a402ca70 con 0x7f10ac06f620 2023-07-29T13:27:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:27:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 10 monclient: dump: 2023-07-29T13:27:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:19.548 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 10 monclient: _finish_auth 0 2023-07-29T13:27:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 10 monclient: _check_auth_tickets 2023-07-29T13:27:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 1 -- 172.21.15.45:0/8804708 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a402cbd0 con 0x7f10ac06f620 2023-07-29T13:27:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10b28bc640 5 monclient: authenticate success, global_id 4111 2023-07-29T13:27:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10b28bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:27:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10b28bc640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 1 -- 172.21.15.45:0/8804708 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a40343f0 con 0x7f10ac06f620 2023-07-29T13:27:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:19.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10b28bc640 4 set_mon_vals no callback set 2023-07-29T13:27:19.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:19.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.545+0000 7f10aaffd640 10 monclient: dump: 2023-07-29T13:27:19.550 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:19.550 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:19.550 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:19.550 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:19.550 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:19.551 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:19.552 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:19.547+0000 7f9c90e32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-29T13:27:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 10 monclient: shutdown 2023-07-29T13:27:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 1 -- 172.21.15.45:0/8804708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ac06f620 msgr2=0x7f10ac1491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 1 --2- 172.21.15.45:0/8804708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ac06f620 0x7f10ac1491e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009e50 tx=0x7f10a402bed0 comp rx=0 tx=0).stop 2023-07-29T13:27:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 1 -- 172.21.15.45:0/8804708 shutdown_connections 2023-07-29T13:27:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 1 --2- 172.21.15.45:0/8804708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ac06f620 0x7f10ac1491e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 1 --2- 172.21.15.45:0/8804708 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f10ac142490 0x7f10ac06f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 1 -- 172.21.15.45:0/8804708 >> 172.21.15.45:0/8804708 conn(0x7f10ac0b4bf0 msgr2=0x7f10ac0b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 1 -- 172.21.15.45:0/8804708 shutdown_connections 2023-07-29T13:27:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 1 -- 172.21.15.45:0/8804708 wait complete. 2023-07-29T13:27:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 10 monclient: build_initial_monmap 2023-07-29T13:27:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.546+0000 7f10b28bc640 10 monclient: monmap: 2023-07-29T13:27:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:19.557 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2023-07-29T13:27:19.557 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2023-07-29T13:27:19.557 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:19.557 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:19.557 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:19.557 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:19.557 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:19.557 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:19.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.547+0000 7f10b28bc640 1 librados: starting msgr at 2023-07-29T13:27:19.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.547+0000 7f10b28bc640 1 librados: starting objecter 2023-07-29T13:27:19.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.547+0000 7f10b28bc640 1 Processor -- start 2023-07-29T13:27:19.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.547+0000 7f10b28bc640 1 -- start start 2023-07-29T13:27:19.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.547+0000 7f10b28bc640 1 librados: setting wanted keys 2023-07-29T13:27:19.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.547+0000 7f10b28bc640 1 librados: calling monclient init 2023-07-29T13:27:19.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.547+0000 7f10b28bc640 10 monclient: init 2023-07-29T13:27:19.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:19.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T13:27:19.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10ac06f620 0x7f10ac0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:19.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 10 monclient(hunting): picked mon.noname-a con 0x7f10ac06f620 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:19.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ac1420c0 0x7f10ac138c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:19.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 10 monclient(hunting): picked mon.noname-b con 0x7f10ac1420c0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:19.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f10ac142490 0x7f10ac139150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:19.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 10 monclient(hunting): picked mon.noname-c con 0x7f10ac142490 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:19.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:19.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac139690 con 0x7f10ac06f620 2023-07-29T13:27:19.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:19.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac06b010 con 0x7f10ac1420c0 2023-07-29T13:27:19.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:19.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac11cba0 con 0x7f10ac142490 2023-07-29T13:27:19.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:19.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b28bc640 10 monclient(hunting): authenticate will time out at 1362.775333s 2023-07-29T13:27:19.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10b0e59640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f10ac142490 0x7f10ac139150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:19.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.548+0000 7f10ab7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ac1420c0 0x7f10ac138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:19.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10b0e59640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f10ac142490 0x7f10ac139150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.45:54368/0 (socket says 172.21.15.45:54368) 2023-07-29T13:27:19.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10b0e59640 1 -- 172.21.15.45:0/912090618 learned_addr learned my addr 172.21.15.45:0/912090618 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-07-29T13:27:19.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10b0e59640 10 monclient(hunting): get_auth_request con 0x7f10ac142490 auth_method 0 2023-07-29T13:27:19.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10b0e59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:19.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10b0e59640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:19.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10b0e59640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:19.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10ab7fe640 10 monclient(hunting): get_auth_request con 0x7f10ac1420c0 auth_method 0 2023-07-29T13:27:19.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:19.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10ab7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:19.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10ab7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:19.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10b0e59640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:19.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10b0e59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:19.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10b0e59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:19.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:19.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:19.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.549+0000 7f10ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:19.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.550+0000 7f10abfff640 1 --2- 172.21.15.45:0/912090618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10ac06f620 0x7f10ac0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:19.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.551+0000 7f10abfff640 10 monclient(hunting): get_auth_request con 0x7f10ac06f620 auth_method 0 2023-07-29T13:27:19.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.551+0000 7f10abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:19.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.551+0000 7f10abfff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:19.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.551+0000 7f10abfff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:19.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.551+0000 7f10abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:19.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.551+0000 7f10abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:19.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.551+0000 7f10abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:19.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.552+0000 7f10ab7fe640 10 monclient(hunting): handle_auth_done global_id 4114 payload 824 2023-07-29T13:27:19.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.552+0000 7f10ab7fe640 1 -- 172.21.15.45:0/912090618 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f10ac142490 msgr2=0x7f10ac139150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:19.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.552+0000 7f10ab7fe640 1 --2- 172.21.15.45:0/912090618 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f10ac142490 0x7f10ac139150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:19.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.552+0000 7f10ab7fe640 1 -- 172.21.15.45:0/912090618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10ac06f620 msgr2=0x7f10ac0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:19.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.552+0000 7f10ab7fe640 1 --2- 172.21.15.45:0/912090618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10ac06f620 0x7f10ac0adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:19.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.552+0000 7f10ab7fe640 10 monclient: _finish_hunting 0 2023-07-29T13:27:19.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.552+0000 7f10ab7fe640 1 monclient: found mon.noname-b 2023-07-29T13:27:19.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.552+0000 7f10ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:19.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.552+0000 7f10ab7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:27:19.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.552+0000 7f10ab7fe640 1 -- 172.21.15.45:0/912090618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a4009b30 con 0x7f10ac1420c0 2023-07-29T13:27:19.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.552+0000 7f10b0e59640 1 --2- 172.21.15.45:0/912090618 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f10ac142490 0x7f10ac139150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-29T13:27:19.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.554+0000 7f10ab7fe640 1 --2- 172.21.15.45:0/912090618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ac1420c0 0x7f10ac138c10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f10a000b410 tx=0x7f10a000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:19.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.554+0000 7f10a8ff9640 10 client.?.objecter ms_handle_connect 0x7f10ac1420c0 2023-07-29T13:27:19.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.554+0000 7f10a8ff9640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:19.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 1 -- 172.21.15.45:0/912090618 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0011830 con 0x7f10ac1420c0 2023-07-29T13:27:19.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:19.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:27:19.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 10 monclient: dump: 2023-07-29T13:27:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:19.569 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:19.569 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:19.569 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:19.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 10 monclient: _finish_auth 0 2023-07-29T13:27:19.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 10 monclient: _check_auth_tickets 2023-07-29T13:27:19.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:19.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 1 -- 172.21.15.45:0/912090618 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a00119b0 con 0x7f10ac1420c0 2023-07-29T13:27:19.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:19.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 1 -- 172.21.15.45:0/912090618 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0011d60 con 0x7f10ac1420c0 2023-07-29T13:27:19.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:19.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:19.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10a8ff9640 10 monclient: dump: 2023-07-29T13:27:19.570 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:19.570 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:19.570 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:19.570 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:19.571 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:19.571 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:19.571 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:19.571 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:19.571 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:19.571 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:19.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10b28bc640 5 monclient: authenticate success, global_id 4114 2023-07-29T13:27:19.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10b28bc640 10 monclient: _renew_subs 2023-07-29T13:27:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10b28bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10b28bc640 1 -- 172.21.15.45:0/912090618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac0ae110 con 0x7f10ac1420c0 2023-07-29T13:27:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10aa7fc640 4 set_mon_vals no callback set 2023-07-29T13:27:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10b28bc640 10 client.4114.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10b28bc640 10 monclient: _renew_subs 2023-07-29T13:27:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10b28bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10b28bc640 1 -- 172.21.15.45:0/912090618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac06b9b0 con 0x7f10ac1420c0 2023-07-29T13:27:19.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.559+0000 7f10b28bc640 1 librados: init done 2023-07-29T13:27:19.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.560+0000 7f10b28bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:19.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.560+0000 7f10b28bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:19.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.560+0000 7f10b28bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:19.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.560+0000 7f10b28bc640 1 -- 172.21.15.45:0/912090618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac11ca10 con 0x7f10ac1420c0 2023-07-29T13:27:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.705+0000 7f10a8ff9640 1 -- 172.21.15.45:0/912090618 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f10a0024740 con 0x7f10ac1420c0 2023-07-29T13:27:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.705+0000 7f10a8ff9640 1 -- 172.21.15.45:0/912090618 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f10a001a700 con 0x7f10ac1420c0 2023-07-29T13:27:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.705+0000 7f10a8ff9640 10 client.4114.objecter ms_dispatch 0x7f10ac000b90 osd_map(2..2 src has 1..2) v4 2023-07-29T13:27:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.705+0000 7f10a8ff9640 3 client.4114.objecter handle_osd_map got epochs [2,2] > 0 2023-07-29T13:27:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.705+0000 7f10a8ff9640 3 client.4114.objecter handle_osd_map decoding full epoch 2 2023-07-29T13:27:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.705+0000 7f10a8ff9640 20 client.4114.objecter dump_active .. 0 homeless 2023-07-29T13:27:19.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.705+0000 7f10a8ff9640 1 -- 172.21.15.45:0/912090618 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f10a0039b00 con 0x7f10ac1420c0 2023-07-29T13:27:19.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.705+0000 7f10a8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:19.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.705+0000 7f10a8ff9640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.779+0000 7f10b28bc640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "6069a944-26b4-4b00-94e2-f91c485e8957", "id": 0}] 2023-07-29T13:27:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.779+0000 7f10b28bc640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "6069a944-26b4-4b00-94e2-f91c485e8957", "id": 0}] 2023-07-29T13:27:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.779+0000 7f10b28bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.779+0000 7f10b28bc640 1 -- 172.21.15.45:0/912090618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6069a944-26b4-4b00-94e2-f91c485e8957", "id": 0} v 0) v1 -- 0x7f10ac070820 con 0x7f10ac1420c0 2023-07-29T13:27:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.937+0000 7f10a8ff9640 1 -- 172.21.15.45:0/912090618 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6069a944-26b4-4b00-94e2-f91c485e8957", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f10a0028030 con 0x7f10ac1420c0 2023-07-29T13:27:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.937+0000 7f10a8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "6069a944-26b4-4b00-94e2-f91c485e8957", "id": 0}] 2023-07-29T13:27:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.937+0000 7f10a8ff9640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:19.939 INFO:teuthology.orchestra.run.smithi045.stdout:0 2023-07-29T13:27:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 10 librados: watch_flush enter 2023-07-29T13:27:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 10 librados: watch_flush exit 2023-07-29T13:27:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 20 client.4114.objecter shutdown clearing up homeless session... 2023-07-29T13:27:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 10 client.4114.objecter successfully canceled tick 2023-07-29T13:27:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 10 monclient: shutdown 2023-07-29T13:27:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 1 -- 172.21.15.45:0/912090618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ac1420c0 msgr2=0x7f10ac138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:19.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 1 --2- 172.21.15.45:0/912090618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ac1420c0 0x7f10ac138c10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f10a000b410 tx=0x7f10a000b8d0 comp rx=0 tx=0).stop 2023-07-29T13:27:19.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 1 -- 172.21.15.45:0/912090618 shutdown_connections 2023-07-29T13:27:19.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 1 --2- 172.21.15.45:0/912090618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ac1420c0 0x7f10ac138c10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:19.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 1 --2- 172.21.15.45:0/912090618 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f10ac142490 0x7f10ac139150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:19.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 1 --2- 172.21.15.45:0/912090618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10ac06f620 0x7f10ac0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:19.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.938+0000 7f10b28bc640 1 -- 172.21.15.45:0/912090618 >> 172.21.15.45:0/912090618 conn(0x7f10ac0b4bf0 msgr2=0x7f10ac13a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:19.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.939+0000 7f10b28bc640 1 -- 172.21.15.45:0/912090618 shutdown_connections 2023-07-29T13:27:19.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.939+0000 7f10b28bc640 1 -- 172.21.15.45:0/912090618 wait complete. 2023-07-29T13:27:19.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:19.939+0000 7f10b28bc640 1 librados: shutdown 2023-07-29T13:27:19.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new ab18edd4-e1cb-4815-b5a8-af2b4b2e790b 1 2023-07-29T13:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.118+0000 7f726ea48640 10 monclient: get_monmap_and_config 2023-07-29T13:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.118+0000 7f726ea48640 10 monclient: build_initial_monmap 2023-07-29T13:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.118+0000 7f726ea48640 10 monclient: monmap: 2023-07-29T13:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:27:20.119001+0000 2023-07-29T13:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:27:20.119001+0000 2023-07-29T13:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:20.121 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 1 Processor -- start 2023-07-29T13:27:20.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 1 -- start start 2023-07-29T13:27:20.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 10 monclient: init 2023-07-29T13:27:20.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2023-07-29T13:27:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7268065b90 addr v1:172.21.15.40:6789/0 2023-07-29T13:27:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f72680aa960 0x7f72680a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 10 monclient(hunting): picked mon.noname-b con 0x7f72680aa960 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7268140da0 0x7f726814b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 10 monclient(hunting): picked mon.noname-a con 0x7f7268140da0 addr v2:172.21.15.40:3300/0 2023-07-29T13:27:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7268065f60 con 0x7f7268065b90 2023-07-29T13:27:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72680aa540 con 0x7f7268140da0 2023-07-29T13:27:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726813e7f0 con 0x7f72680aa960 2023-07-29T13:27:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.119+0000 7f726ea48640 10 monclient(hunting): authenticate will time out at 1363.346871s 2023-07-29T13:27:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.120+0000 7f72677fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f72680aa960 0x7f72680a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.120+0000 7f726cfe5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7268140da0 0x7f726814b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.120+0000 7f7267fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7268065b90 0x7f726813e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.45:37826/0 (socket says 172.21.15.45:37826) 2023-07-29T13:27:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.120+0000 7f7267fff640 1 -- 172.21.15.45:0/98816938 learned_addr learned my addr 172.21.15.45:0/98816938 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-07-29T13:27:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.120+0000 7f72677fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f72680aa960 0x7f72680a9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.45:54374/0 (socket says 172.21.15.45:54374) 2023-07-29T13:27:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.120+0000 7f72677fe640 10 monclient(hunting): get_auth_request con 0x7f72680aa960 auth_method 0 2023-07-29T13:27:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.120+0000 7f72677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.120+0000 7f72677fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.120+0000 7f72677fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.121+0000 7f726cfe5640 10 monclient(hunting): get_auth_request con 0x7f7268140da0 auth_method 0 2023-07-29T13:27:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.121+0000 7f726cfe5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.121+0000 7f726cfe5640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.121+0000 7f726cfe5640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.121+0000 7f72677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.121+0000 7f72677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:20.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.121+0000 7f72677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:20.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.122+0000 7f726cfe5640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:20.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.122+0000 7f726cfe5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:20.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.122+0000 7f726cfe5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:20.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.123+0000 7f72677fe640 10 monclient(hunting): handle_auth_done global_id 4109 payload 274 2023-07-29T13:27:20.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.123+0000 7f72677fe640 1 -- 172.21.15.45:0/98816938 >> v2:172.21.15.40:3300/0 conn(0x7f7268140da0 msgr2=0x7f726814b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:20.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.123+0000 7f72677fe640 1 --2- 172.21.15.45:0/98816938 >> v2:172.21.15.40:3300/0 conn(0x7f7268140da0 0x7f726814b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.123+0000 7f72677fe640 1 -- 172.21.15.45:0/98816938 >> v1:172.21.15.40:6789/0 conn(0x7f7268065b90 legacy=0x7f726813e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:20.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.123+0000 7f72677fe640 10 monclient: _finish_hunting 0 2023-07-29T13:27:20.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.123+0000 7f72677fe640 1 monclient: found mon.noname-b 2023-07-29T13:27:20.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.123+0000 7f72677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.123+0000 7f72677fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:27:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.123+0000 7f72677fe640 1 -- 172.21.15.45:0/98816938 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72680a9720 con 0x7f72680aa960 2023-07-29T13:27:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.126+0000 7f72677fe640 1 --2- 172.21.15.45:0/98816938 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f72680aa960 0x7f72680a9db0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f725c0097a0 tx=0x7f725c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ad7e119da99155b server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 1 -- 172.21.15.45:0/98816938 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c02e070 con 0x7f72680aa960 2023-07-29T13:27:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:27:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 10 monclient: dump: 2023-07-29T13:27:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:20.133 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:20.133 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 10 monclient: _finish_auth 0 2023-07-29T13:27:20.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 10 monclient: _check_auth_tickets 2023-07-29T13:27:20.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:20.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 1 -- 172.21.15.45:0/98816938 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c0315a0 con 0x7f72680aa960 2023-07-29T13:27:20.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:20.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 1 -- 172.21.15.45:0/98816938 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c031930 con 0x7f72680aa960 2023-07-29T13:27:20.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 5 monclient: authenticate success, global_id 4109 2023-07-29T13:27:20.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:20.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 4 set_mon_vals no callback set 2023-07-29T13:27:20.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:20.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:27:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f7266ffd640 10 monclient: dump: 2023-07-29T13:27:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:20.136 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:20.136 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:20.136 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 10 monclient: shutdown 2023-07-29T13:27:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 1 -- 172.21.15.45:0/98816938 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f72680aa960 msgr2=0x7f72680a9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 1 --2- 172.21.15.45:0/98816938 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f72680aa960 0x7f72680a9db0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f725c0097a0 tx=0x7f725c02b840 comp rx=0 tx=0).stop 2023-07-29T13:27:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 1 -- 172.21.15.45:0/98816938 shutdown_connections 2023-07-29T13:27:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 1 --2- 172.21.15.45:0/98816938 >> v2:172.21.15.40:3300/0 conn(0x7f7268140da0 0x7f726814b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 1 --2- 172.21.15.45:0/98816938 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f72680aa960 0x7f72680a9db0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 1 -- 172.21.15.45:0/98816938 >> 172.21.15.45:0/98816938 conn(0x7f72680b3b80 msgr2=0x7f72680b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 1 -- 172.21.15.45:0/98816938 shutdown_connections 2023-07-29T13:27:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.130+0000 7f726ea48640 1 -- 172.21.15.45:0/98816938 wait complete. 2023-07-29T13:27:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.131+0000 7f726ea48640 10 monclient: build_initial_monmap 2023-07-29T13:27:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.131+0000 7f726ea48640 10 monclient: monmap: 2023-07-29T13:27:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2023-07-29T13:27:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2023-07-29T13:27:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:20.141 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.131+0000 7f726ea48640 1 librados: starting msgr at 2023-07-29T13:27:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.131+0000 7f726ea48640 1 librados: starting objecter 2023-07-29T13:27:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.131+0000 7f726ea48640 1 Processor -- start 2023-07-29T13:27:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 1 -- start start 2023-07-29T13:27:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 1 librados: setting wanted keys 2023-07-29T13:27:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 1 librados: calling monclient init 2023-07-29T13:27:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 10 monclient: init 2023-07-29T13:27:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:27:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7268065b90 0x7f72680ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 10 monclient(hunting): picked mon.noname-c con 0x7f7268065b90 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72680aa960 0x7f7268143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 10 monclient(hunting): picked mon.noname-b con 0x7f72680aa960 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268140da0 0x7f72681d5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 10 monclient(hunting): picked mon.noname-a con 0x7f7268140da0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72680ad150 con 0x7f7268140da0 2023-07-29T13:27:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72680ad670 con 0x7f72680aa960 2023-07-29T13:27:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72680ad830 con 0x7f7268065b90 2023-07-29T13:27:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.132+0000 7f726ea48640 10 monclient(hunting): authenticate will time out at 1363.359816s 2023-07-29T13:27:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f7267fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7268065b90 0x7f72680ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f72677fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72680aa960 0x7f7268143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f726cfe5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268140da0 0x7f72681d5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f7267fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7268065b90 0x7f72680ac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.45:54384/0 (socket says 172.21.15.45:54384) 2023-07-29T13:27:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f7267fff640 1 -- 172.21.15.45:0/3455404877 learned_addr learned my addr 172.21.15.45:0/3455404877 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-07-29T13:27:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f7267fff640 10 monclient(hunting): get_auth_request con 0x7f7268065b90 auth_method 0 2023-07-29T13:27:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f7267fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:20.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f7267fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:20.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f7267fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:20.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f72677fe640 10 monclient(hunting): get_auth_request con 0x7f72680aa960 auth_method 0 2023-07-29T13:27:20.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f72677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:20.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f72677fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:20.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f72677fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:20.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f726cfe5640 10 monclient(hunting): get_auth_request con 0x7f7268140da0 auth_method 0 2023-07-29T13:27:20.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f726cfe5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f726cfe5640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.133+0000 7f726cfe5640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.134+0000 7f7267fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.134+0000 7f7267fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.134+0000 7f7267fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.134+0000 7f72677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.134+0000 7f72677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.134+0000 7f72677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.134+0000 7f726cfe5640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.134+0000 7f726cfe5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.134+0000 7f726cfe5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.137+0000 7f72677fe640 10 monclient(hunting): handle_auth_done global_id 4117 payload 824 2023-07-29T13:27:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.137+0000 7f72677fe640 1 -- 172.21.15.45:0/3455404877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7268065b90 msgr2=0x7f72680ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.137+0000 7f72677fe640 1 --2- 172.21.15.45:0/3455404877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7268065b90 0x7f72680ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.137+0000 7f72677fe640 1 -- 172.21.15.45:0/3455404877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268140da0 msgr2=0x7f72681d5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.137+0000 7f72677fe640 1 --2- 172.21.15.45:0/3455404877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268140da0 0x7f72681d5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.137+0000 7f72677fe640 10 monclient: _finish_hunting 0 2023-07-29T13:27:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.137+0000 7f72677fe640 1 monclient: found mon.noname-b 2023-07-29T13:27:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.137+0000 7f72677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.137+0000 7f72677fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:27:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.137+0000 7f72677fe640 1 -- 172.21.15.45:0/3455404877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c009400 con 0x7f72680aa960 2023-07-29T13:27:20.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.139+0000 7f72677fe640 1 --2- 172.21.15.45:0/3455404877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72680aa960 0x7f7268143e80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f725c033000 tx=0x7f725c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:20.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.139+0000 7f7264ff9640 10 client.?.objecter ms_handle_connect 0x7f72680aa960 2023-07-29T13:27:20.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.139+0000 7f7264ff9640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:20.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.142+0000 7f7264ff9640 1 -- 172.21.15.45:0/3455404877 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c032840 con 0x7f72680aa960 2023-07-29T13:27:20.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.142+0000 7f7264ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:20.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.142+0000 7f7264ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:27:20.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.142+0000 7f7264ff9640 10 monclient: dump: 2023-07-29T13:27:20.152 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:20.152 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:20.152 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.152 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.152 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:20.152 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.152 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:20.153 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:20.153 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:20.153 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f7264ff9640 10 monclient: _finish_auth 0 2023-07-29T13:27:20.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f7264ff9640 10 monclient: _check_auth_tickets 2023-07-29T13:27:20.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f7264ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:20.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f7264ff9640 1 -- 172.21.15.45:0/3455404877 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c0329a0 con 0x7f72680aa960 2023-07-29T13:27:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f726ea48640 5 monclient: authenticate success, global_id 4117 2023-07-29T13:27:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f7264ff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f726ea48640 10 monclient: _renew_subs 2023-07-29T13:27:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f7264ff9640 1 -- 172.21.15.45:0/3455404877 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c032d50 con 0x7f72680aa960 2023-07-29T13:27:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f726ea48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f726ea48640 1 -- 172.21.15.45:0/3455404877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72680acc00 con 0x7f72680aa960 2023-07-29T13:27:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f7264ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f7264ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f7264ff9640 10 monclient: dump: 2023-07-29T13:27:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.156 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:20.156 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:20.156 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:20.156 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f72667fc640 4 set_mon_vals no callback set 2023-07-29T13:27:20.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f726ea48640 10 client.4117.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:20.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f726ea48640 10 monclient: _renew_subs 2023-07-29T13:27:20.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f726ea48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f726ea48640 1 -- 172.21.15.45:0/3455404877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72681d6430 con 0x7f72680aa960 2023-07-29T13:27:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.143+0000 7f726ea48640 1 librados: init done 2023-07-29T13:27:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.144+0000 7f726ea48640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.144+0000 7f726ea48640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.144+0000 7f726ea48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.144+0000 7f726ea48640 1 -- 172.21.15.45:0/3455404877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7268141200 con 0x7f72680aa960 2023-07-29T13:27:20.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.248+0000 7f7264ff9640 1 -- 172.21.15.45:0/3455404877 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f725c04e730 con 0x7f72680aa960 2023-07-29T13:27:20.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.248+0000 7f7264ff9640 1 -- 172.21.15.45:0/3455404877 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f725c045910 con 0x7f72680aa960 2023-07-29T13:27:20.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.248+0000 7f7264ff9640 10 client.4117.objecter ms_dispatch 0x7f7268000b90 osd_map(3..3 src has 1..3) v4 2023-07-29T13:27:20.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.248+0000 7f7264ff9640 3 client.4117.objecter handle_osd_map got epochs [3,3] > 0 2023-07-29T13:27:20.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.248+0000 7f7264ff9640 3 client.4117.objecter handle_osd_map decoding full epoch 3 2023-07-29T13:27:20.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.248+0000 7f7264ff9640 20 client.4117.objecter dump_active .. 0 homeless 2023-07-29T13:27:20.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.248+0000 7f7264ff9640 1 -- 172.21.15.45:0/3455404877 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f725c063b00 con 0x7f72680aa960 2023-07-29T13:27:20.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.248+0000 7f7264ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:20.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.248+0000 7f7264ff9640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:20.285 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:20.284+0000 7ff142a2c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-29T13:27:20.322 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:20.320+0000 7f9c90e32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-29T13:27:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.321+0000 7f726ea48640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "ab18edd4-e1cb-4815-b5a8-af2b4b2e790b", "id": 1}] 2023-07-29T13:27:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.321+0000 7f726ea48640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "ab18edd4-e1cb-4815-b5a8-af2b4b2e790b", "id": 1}] 2023-07-29T13:27:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.321+0000 7f726ea48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.321+0000 7f726ea48640 1 -- 172.21.15.45:0/3455404877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ab18edd4-e1cb-4815-b5a8-af2b4b2e790b", "id": 1} v 0) v1 -- 0x7f72680ae1e0 con 0x7f72680aa960 2023-07-29T13:27:20.342 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:20.341+0000 7ff142a2c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-29T13:27:20.389 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:20.388+0000 7ff142a2c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-29T13:27:20.395 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:20.393+0000 7f9c90e32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-29T13:27:20.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.437+0000 7f7264ff9640 1 -- 172.21.15.45:0/3455404877 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ab18edd4-e1cb-4815-b5a8-af2b4b2e790b", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f725c052070 con 0x7f72680aa960 2023-07-29T13:27:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.437+0000 7f7264ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "ab18edd4-e1cb-4815-b5a8-af2b4b2e790b", "id": 1}] 2023-07-29T13:27:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.437+0000 7f7264ff9640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:20.439 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-07-29T13:27:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 10 librados: watch_flush enter 2023-07-29T13:27:20.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 10 librados: watch_flush exit 2023-07-29T13:27:20.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 20 client.4117.objecter shutdown clearing up homeless session... 2023-07-29T13:27:20.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 10 client.4117.objecter successfully canceled tick 2023-07-29T13:27:20.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 10 monclient: shutdown 2023-07-29T13:27:20.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:20.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 1 -- 172.21.15.45:0/3455404877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72680aa960 msgr2=0x7f7268143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:20.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 1 --2- 172.21.15.45:0/3455404877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72680aa960 0x7f7268143e80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f725c033000 tx=0x7f725c0046c0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 1 -- 172.21.15.45:0/3455404877 shutdown_connections 2023-07-29T13:27:20.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 1 --2- 172.21.15.45:0/3455404877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72680aa960 0x7f7268143e80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 1 --2- 172.21.15.45:0/3455404877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268140da0 0x7f72681d5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 1 --2- 172.21.15.45:0/3455404877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7268065b90 0x7f72680ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 1 -- 172.21.15.45:0/3455404877 >> 172.21.15.45:0/3455404877 conn(0x7f72680b3b80 msgr2=0x7f726813deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 1 -- 172.21.15.45:0/3455404877 shutdown_connections 2023-07-29T13:27:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 1 -- 172.21.15.45:0/3455404877 wait complete. 2023-07-29T13:27:20.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.438+0000 7f726ea48640 1 librados: shutdown 2023-07-29T13:27:20.448 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:20.446+0000 7f9c90e32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-29T13:27:20.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new 4575bc2c-4d43-46d3-95ba-1dc2c94278e6 2 2023-07-29T13:27:20.507 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:20.506+0000 7ff142a2c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-29T13:27:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.554+0000 7fb892da6640 10 monclient: get_monmap_and_config 2023-07-29T13:27:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.554+0000 7fb892da6640 10 monclient: build_initial_monmap 2023-07-29T13:27:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.554+0000 7fb892da6640 10 monclient: monmap: 2023-07-29T13:27:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:27:20.554716+0000 2023-07-29T13:27:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:27:20.554716+0000 2023-07-29T13:27:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:20.557 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.554+0000 7fb892da6640 1 Processor -- start 2023-07-29T13:27:20.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.554+0000 7fb892da6640 1 -- start start 2023-07-29T13:27:20.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.554+0000 7fb892da6640 10 monclient: init 2023-07-29T13:27:20.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 10 monclient: _add_conns ranks=[1,3,0,4,2] 2023-07-29T13:27:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb88c0b3ae0 addr v1:172.21.15.45:6789/0 2023-07-29T13:27:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb88c13e3a0 0x7fb88c140c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 10 monclient(hunting): picked mon.noname-a con 0x7fb88c13e3a0 addr v2:172.21.15.40:3300/0 2023-07-29T13:27:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb88c0a8910 addr v1:172.21.15.40:6789/0 2023-07-29T13:27:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88c0b3eb0 con 0x7fb88c0a8910 2023-07-29T13:27:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88c0a9950 con 0x7fb88c0b3ae0 2023-07-29T13:27:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88c0b0f10 con 0x7fb88c13e3a0 2023-07-29T13:27:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.555+0000 7fb892da6640 10 monclient(hunting): authenticate will time out at 1363.782853s 2023-07-29T13:27:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.556+0000 7fb890b42640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb88c0b3ae0 0x7fb88c13dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52858/0 (socket says 172.21.15.45:52858) 2023-07-29T13:27:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.556+0000 7fb890b42640 1 -- 172.21.15.45:0/279736209 learned_addr learned my addr 172.21.15.45:0/279736209 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-07-29T13:27:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.558+0000 7fb88b7fe640 1 -- 172.21.15.45:0/279736209 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071751554 0 0) 0x7fb88c0a9950 con 0x7fb88c0b3ae0 2023-07-29T13:27:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.558+0000 7fb88b7fe640 10 monclient(hunting): _negotiate 2023-07-29T13:27:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.558+0000 7fb88b7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.558+0000 7fb88b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.558+0000 7fb88b7fe640 10 monclient(hunting): my global_id is 4120 2023-07-29T13:27:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.558+0000 7fb88b7fe640 1 -- 172.21.15.45:0/279736209 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb87c003520 con 0x7fb88c0b3ae0 2023-07-29T13:27:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.559+0000 7fb88bfff640 1 --2- 172.21.15.45:0/279736209 >> v2:172.21.15.40:3300/0 conn(0x7fb88c13e3a0 0x7fb88c140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.560+0000 7fb88bfff640 10 monclient(hunting): get_auth_request con 0x7fb88c13e3a0 auth_method 0 2023-07-29T13:27:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.560+0000 7fb88bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.560+0000 7fb88bfff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.560+0000 7fb88bfff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.561+0000 7fb88bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.561+0000 7fb88bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.561+0000 7fb88bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:20.565 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:20.563+0000 7f9c90e32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-29T13:27:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.567+0000 7fb88bfff640 10 monclient(hunting): handle_auth_done global_id 4119 payload 274 2023-07-29T13:27:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.567+0000 7fb88bfff640 1 -- 172.21.15.45:0/279736209 >> v1:172.21.15.45:6789/0 conn(0x7fb88c0b3ae0 legacy=0x7fb88c13dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.567+0000 7fb88bfff640 1 -- 172.21.15.45:0/279736209 >> v1:172.21.15.40:6789/0 conn(0x7fb88c0a8910 legacy=0x7fb88c14b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.567+0000 7fb88bfff640 10 monclient: _finish_hunting 0 2023-07-29T13:27:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.567+0000 7fb88bfff640 1 monclient: found mon.noname-a 2023-07-29T13:27:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.567+0000 7fb88bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.567+0000 7fb88bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:27:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.567+0000 7fb88bfff640 1 -- 172.21.15.45:0/279736209 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c0a9c30 con 0x7fb88c13e3a0 2023-07-29T13:27:20.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.570+0000 7fb88bfff640 1 --2- 172.21.15.45:0/279736209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb88c13e3a0 0x7fb88c140c10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb8780097a0 tx=0x7fb87802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2eb906d8338464 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:20.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 1 -- 172.21.15.45:0/279736209 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878031830 con 0x7fb88c13e3a0 2023-07-29T13:27:20.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:20.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:27:20.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 10 monclient: dump: 2023-07-29T13:27:20.594 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:20.594 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:20.594 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.594 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.594 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:20.594 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.594 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:20.595 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:20.595 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:20.595 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 10 monclient: _finish_auth 0 2023-07-29T13:27:20.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 10 monclient: _check_auth_tickets 2023-07-29T13:27:20.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:20.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 1 -- 172.21.15.45:0/279736209 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb878031990 con 0x7fb88c13e3a0 2023-07-29T13:27:20.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:20.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 1 -- 172.21.15.45:0/279736209 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878031d20 con 0x7fb88c13e3a0 2023-07-29T13:27:20.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:20.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:27:20.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb88b7fe640 10 monclient: dump: 2023-07-29T13:27:20.596 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:20.596 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:20.596 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.597 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.597 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:20.597 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.597 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:20.597 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:20.597 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:20.597 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb892da6640 5 monclient: authenticate success, global_id 4119 2023-07-29T13:27:20.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb892da6640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:20.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.591+0000 7fb892da6640 4 set_mon_vals no callback set 2023-07-29T13:27:20.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.592+0000 7fb892da6640 10 monclient: shutdown 2023-07-29T13:27:20.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.592+0000 7fb892da6640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:20.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.592+0000 7fb892da6640 1 -- 172.21.15.45:0/279736209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb88c13e3a0 msgr2=0x7fb88c140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:20.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.592+0000 7fb892da6640 1 --2- 172.21.15.45:0/279736209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb88c13e3a0 0x7fb88c140c10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb8780097a0 tx=0x7fb87802b8d0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.592+0000 7fb892da6640 1 -- 172.21.15.45:0/279736209 shutdown_connections 2023-07-29T13:27:20.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.592+0000 7fb892da6640 1 --2- 172.21.15.45:0/279736209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb88c13e3a0 0x7fb88c140c10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.592+0000 7fb892da6640 1 -- 172.21.15.45:0/279736209 >> 172.21.15.45:0/279736209 conn(0x7fb88c070fc0 msgr2=0x7fb88c0713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:20.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.592+0000 7fb892da6640 1 -- 172.21.15.45:0/279736209 shutdown_connections 2023-07-29T13:27:20.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.592+0000 7fb892da6640 1 -- 172.21.15.45:0/279736209 wait complete. 2023-07-29T13:27:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.592+0000 7fb892da6640 10 monclient: build_initial_monmap 2023-07-29T13:27:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.592+0000 7fb892da6640 10 monclient: monmap: 2023-07-29T13:27:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:20.602 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2023-07-29T13:27:20.602 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2023-07-29T13:27:20.602 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:20.602 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.602 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:20.602 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:20.602 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:20.603 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 librados: starting msgr at 2023-07-29T13:27:20.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 librados: starting objecter 2023-07-29T13:27:20.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 Processor -- start 2023-07-29T13:27:20.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 -- start start 2023-07-29T13:27:20.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 librados: setting wanted keys 2023-07-29T13:27:20.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 librados: calling monclient init 2023-07-29T13:27:20.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 10 monclient: init 2023-07-29T13:27:20.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:20.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 10 monclient: _add_conns ranks=[1,0,2] 2023-07-29T13:27:20.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88c0a8910 0x7fb88c0afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:20.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 10 monclient(hunting): picked mon.noname-b con 0x7fb88c0a8910 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:20.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb88c0b3ae0 0x7fb88c13ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:20.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 10 monclient(hunting): picked mon.noname-a con 0x7fb88c0b3ae0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:20.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb88c13e3a0 0x7fb88c13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:20.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 10 monclient(hunting): picked mon.noname-c con 0x7fb88c13e3a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:20.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:20.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c1d9910 con 0x7fb88c0b3ae0 2023-07-29T13:27:20.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:20.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0ac450 con 0x7fb88c0a8910 2023-07-29T13:27:20.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:20.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0ac610 con 0x7fb88c13e3a0 2023-07-29T13:27:20.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:20.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.593+0000 7fb892da6640 10 monclient(hunting): authenticate will time out at 1363.821053s 2023-07-29T13:27:20.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.594+0000 7fb890b42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88c0a8910 0x7fb88c0afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:20.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.594+0000 7fb890b42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88c0a8910 0x7fb88c0afb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47730/0 (socket says 172.21.15.45:47730) 2023-07-29T13:27:20.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.594+0000 7fb890b42640 1 -- 172.21.15.45:0/4288486888 learned_addr learned my addr 172.21.15.45:0/4288486888 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-07-29T13:27:20.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.594+0000 7fb890b42640 10 monclient(hunting): get_auth_request con 0x7fb88c0a8910 auth_method 0 2023-07-29T13:27:20.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.594+0000 7fb890b42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:20.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.594+0000 7fb890b42640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:20.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.594+0000 7fb890b42640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:20.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.595+0000 7fb88bfff640 1 --2- 172.21.15.45:0/4288486888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb88c0b3ae0 0x7fb88c13ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:20.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.595+0000 7fb890b42640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:20.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.595+0000 7fb890b42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:20.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.595+0000 7fb890b42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:20.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.595+0000 7fb88bfff640 10 monclient(hunting): get_auth_request con 0x7fb88c0b3ae0 auth_method 0 2023-07-29T13:27:20.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.595+0000 7fb88bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:20.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.595+0000 7fb88bfff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:20.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.595+0000 7fb88bfff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:20.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.596+0000 7fb88bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:20.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.596+0000 7fb88bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:20.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.596+0000 7fb88bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:20.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.598+0000 7fb890b42640 10 monclient(hunting): handle_auth_done global_id 4123 payload 824 2023-07-29T13:27:20.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.598+0000 7fb890b42640 1 -- 172.21.15.45:0/4288486888 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb88c13e3a0 msgr2=0x7fb88c13b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:20.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.598+0000 7fb890b42640 1 --2- 172.21.15.45:0/4288486888 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb88c13e3a0 0x7fb88c13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.598+0000 7fb890b42640 1 -- 172.21.15.45:0/4288486888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb88c0b3ae0 msgr2=0x7fb88c13ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:20.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.598+0000 7fb890b42640 1 --2- 172.21.15.45:0/4288486888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb88c0b3ae0 0x7fb88c13ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.598+0000 7fb890b42640 10 monclient: _finish_hunting 0 2023-07-29T13:27:20.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.598+0000 7fb890b42640 1 monclient: found mon.noname-b 2023-07-29T13:27:20.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.598+0000 7fb890b42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:20.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.598+0000 7fb890b42640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:27:20.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.598+0000 7fb890b42640 1 -- 172.21.15.45:0/4288486888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878009400 con 0x7fb88c0a8910 2023-07-29T13:27:20.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.600+0000 7fb890b42640 1 --2- 172.21.15.45:0/4288486888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88c0a8910 0x7fb88c0afb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c500 tx=0x7fb88000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:20.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.600+0000 7fb8897fa640 10 client.?.objecter ms_handle_connect 0x7fb88c0a8910 2023-07-29T13:27:20.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.600+0000 7fb8897fa640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:20.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 1 -- 172.21.15.45:0/4288486888 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880005ce0 con 0x7fb88c0a8910 2023-07-29T13:27:20.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:20.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:27:20.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 10 monclient: dump: 2023-07-29T13:27:20.611 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:20.612 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:20.612 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.612 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.612 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:20.612 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.612 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:20.612 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:20.613 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 10 monclient: _finish_auth 0 2023-07-29T13:27:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 10 monclient: _check_auth_tickets 2023-07-29T13:27:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 1 -- 172.21.15.45:0/4288486888 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb880005e40 con 0x7fb88c0a8910 2023-07-29T13:27:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 1 -- 172.21.15.45:0/4288486888 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880007330 con 0x7fb88c0a8910 2023-07-29T13:27:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb8897fa640 10 monclient: dump: 2023-07-29T13:27:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:20.615 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb892da6640 5 monclient: authenticate success, global_id 4123 2023-07-29T13:27:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb88affd640 4 set_mon_vals no callback set 2023-07-29T13:27:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb892da6640 10 monclient: _renew_subs 2023-07-29T13:27:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb892da6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb892da6640 1 -- 172.21.15.45:0/4288486888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c1d8cd0 con 0x7fb88c0a8910 2023-07-29T13:27:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb892da6640 10 client.4123.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb892da6640 10 monclient: _renew_subs 2023-07-29T13:27:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb892da6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb892da6640 1 -- 172.21.15.45:0/4288486888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c1d9160 con 0x7fb88c0a8910 2023-07-29T13:27:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.604+0000 7fb892da6640 1 librados: init done 2023-07-29T13:27:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.605+0000 7fb892da6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.605+0000 7fb892da6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.605+0000 7fb892da6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.605+0000 7fb892da6640 1 -- 172.21.15.45:0/4288486888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c1d9a80 con 0x7fb88c0a8910 2023-07-29T13:27:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.607+0000 7fb8897fa640 1 -- 172.21.15.45:0/4288486888 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb88001c440 con 0x7fb88c0a8910 2023-07-29T13:27:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.607+0000 7fb8897fa640 1 -- 172.21.15.45:0/4288486888 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fb88001b7d0 con 0x7fb88c0a8910 2023-07-29T13:27:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.607+0000 7fb8897fa640 10 client.4123.objecter ms_dispatch 0x7fb88c000b90 osd_map(4..4 src has 1..4) v4 2023-07-29T13:27:20.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.607+0000 7fb8897fa640 3 client.4123.objecter handle_osd_map got epochs [4,4] > 0 2023-07-29T13:27:20.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.607+0000 7fb8897fa640 3 client.4123.objecter handle_osd_map decoding full epoch 4 2023-07-29T13:27:20.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.608+0000 7fb8897fa640 20 client.4123.objecter dump_active .. 0 homeless 2023-07-29T13:27:20.654 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:20.653+0000 7ff142a2c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-29T13:27:20.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.711+0000 7fb8897fa640 1 -- 172.21.15.45:0/4288486888 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb880017070 con 0x7fb88c0a8910 2023-07-29T13:27:20.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.711+0000 7fb8897fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:20.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.711+0000 7fb8897fa640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:20.719 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:20.718+0000 7f9c90e32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-29T13:27:20.756 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:20.755+0000 7ff142a2c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-29T13:27:20.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.787+0000 7fb892da6640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "4575bc2c-4d43-46d3-95ba-1dc2c94278e6", "id": 2}] 2023-07-29T13:27:20.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.787+0000 7fb892da6640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "4575bc2c-4d43-46d3-95ba-1dc2c94278e6", "id": 2}] 2023-07-29T13:27:20.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.787+0000 7fb892da6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:20.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.787+0000 7fb892da6640 1 -- 172.21.15.45:0/4288486888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4575bc2c-4d43-46d3-95ba-1dc2c94278e6", "id": 2} v 0) v1 -- 0x7fb88c145030 con 0x7fb88c0a8910 2023-07-29T13:27:20.800 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:20.799+0000 7ff142a2c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-29T13:27:20.821 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:20.819+0000 7f9c90e32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-29T13:27:20.867 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:20.865+0000 7f9c90e32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-29T13:27:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.875+0000 7fb8897fa640 1 -- 172.21.15.45:0/4288486888 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4575bc2c-4d43-46d3-95ba-1dc2c94278e6", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb88001ba00 con 0x7fb88c0a8910 2023-07-29T13:27:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.875+0000 7fb8897fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "4575bc2c-4d43-46d3-95ba-1dc2c94278e6", "id": 2}] 2023-07-29T13:27:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.875+0000 7fb8897fa640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:20.878 INFO:teuthology.orchestra.run.smithi045.stdout:2 2023-07-29T13:27:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 10 librados: watch_flush enter 2023-07-29T13:27:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 10 librados: watch_flush exit 2023-07-29T13:27:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 20 client.4123.objecter shutdown clearing up homeless session... 2023-07-29T13:27:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 10 client.4123.objecter successfully canceled tick 2023-07-29T13:27:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 10 monclient: shutdown 2023-07-29T13:27:20.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:20.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 1 -- 172.21.15.45:0/4288486888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88c0a8910 msgr2=0x7fb88c0afb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:20.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 1 --2- 172.21.15.45:0/4288486888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88c0a8910 0x7fb88c0afb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c500 tx=0x7fb88000c9c0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 1 -- 172.21.15.45:0/4288486888 shutdown_connections 2023-07-29T13:27:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 1 --2- 172.21.15.45:0/4288486888 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb88c13e3a0 0x7fb88c13b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 1 --2- 172.21.15.45:0/4288486888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb88c0b3ae0 0x7fb88c13ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 1 --2- 172.21.15.45:0/4288486888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88c0a8910 0x7fb88c0afb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 1 -- 172.21.15.45:0/4288486888 >> 172.21.15.45:0/4288486888 conn(0x7fb88c070fc0 msgr2=0x7fb88c0b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 1 -- 172.21.15.45:0/4288486888 shutdown_connections 2023-07-29T13:27:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 1 -- 172.21.15.45:0/4288486888 wait complete. 2023-07-29T13:27:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.876+0000 7fb892da6640 1 librados: shutdown 2023-07-29T13:27:20.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new e7224640-7d33-4bb7-81a5-99fc26da7763 3 2023-07-29T13:27:20.947 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:20.946+0000 7ff142a2c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-29T13:27:20.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.991+0000 7efd65206640 10 monclient: get_monmap_and_config 2023-07-29T13:27:20.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.991+0000 7efd65206640 10 monclient: build_initial_monmap 2023-07-29T13:27:20.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.991+0000 7efd65206640 10 monclient: monmap: 2023-07-29T13:27:20.993 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:20.993 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:20.993 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:27:20.991856+0000 2023-07-29T13:27:20.993 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:27:20.991856+0000 2023-07-29T13:27:20.993 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:20.993 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:20.993 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:20.994 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:20.994 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:20.994 INFO:teuthology.orchestra.run.smithi045.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:20.994 INFO:teuthology.orchestra.run.smithi045.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:20.994 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:20.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.991+0000 7efd65206640 1 Processor -- start 2023-07-29T13:27:20.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.991+0000 7efd65206640 1 -- start start 2023-07-29T13:27:20.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.991+0000 7efd65206640 10 monclient: init 2023-07-29T13:27:20.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:20.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2023-07-29T13:27:20.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efd60065b90 0x7efd600a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:20.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 10 monclient(hunting): picked mon.noname-c con 0x7efd60065b90 addr v2:172.21.15.45:3300/0 2023-07-29T13:27:20.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efd600aa960 addr v1:172.21.15.45:6789/0 2023-07-29T13:27:20.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efd60140da0 addr v1:172.21.15.40:6789/0 2023-07-29T13:27:20.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd60065f60 con 0x7efd60140da0 2023-07-29T13:27:20.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd600a9720 con 0x7efd600aa960 2023-07-29T13:27:20.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:20.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd600aa540 con 0x7efd60065b90 2023-07-29T13:27:20.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:20.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.992+0000 7efd65206640 10 monclient(hunting): authenticate will time out at 1364.219667s 2023-07-29T13:27:20.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.993+0000 7efd5f59e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efd60140da0 0x7efd6014b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.45:37846/0 (socket says 172.21.15.45:37846) 2023-07-29T13:27:20.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.993+0000 7efd5f59e640 1 -- 172.21.15.45:0/430058932 learned_addr learned my addr 172.21.15.45:0/430058932 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-07-29T13:27:20.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.993+0000 7efd5ed9d640 1 --2- 172.21.15.45:0/430058932 >> v2:172.21.15.45:3300/0 conn(0x7efd60065b90 0x7efd600a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:20.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.993+0000 7efd5ed9d640 10 monclient(hunting): get_auth_request con 0x7efd60065b90 auth_method 0 2023-07-29T13:27:20.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.993+0000 7efd5ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:20.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.993+0000 7efd5ed9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:20.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.993+0000 7efd5ed9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:20.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.994+0000 7efd5ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:20.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.994+0000 7efd5ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:20.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.994+0000 7efd5ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.995+0000 7efd5e59c640 1 -- 172.21.15.45:0/430058932 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710487813 0 0) 0x7efd60065f60 con 0x7efd60140da0 2023-07-29T13:27:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.995+0000 7efd5e59c640 10 monclient(hunting): _negotiate 2023-07-29T13:27:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.995+0000 7efd5e59c640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.995+0000 7efd5e59c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.995+0000 7efd5e59c640 10 monclient(hunting): my global_id is 4128 2023-07-29T13:27:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.995+0000 7efd5e59c640 1 -- 172.21.15.45:0/430058932 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd4c003520 con 0x7efd60140da0 2023-07-29T13:27:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.996+0000 7efd5ed9d640 10 monclient(hunting): handle_auth_done global_id 4126 payload 274 2023-07-29T13:27:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.996+0000 7efd5ed9d640 1 -- 172.21.15.45:0/430058932 >> v1:172.21.15.45:6789/0 conn(0x7efd600aa960 legacy=0x7efd6013e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.996+0000 7efd5ed9d640 1 -- 172.21.15.45:0/430058932 >> v1:172.21.15.40:6789/0 conn(0x7efd60140da0 legacy=0x7efd6014b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.996+0000 7efd5ed9d640 10 monclient: _finish_hunting 0 2023-07-29T13:27:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.996+0000 7efd5ed9d640 1 monclient: found mon.noname-c 2023-07-29T13:27:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.996+0000 7efd5ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:21.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.996+0000 7efd5ed9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:27:21.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:20.996+0000 7efd5ed9d640 1 -- 172.21.15.45:0/430058932 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd600a9990 con 0x7efd60065b90 2023-07-29T13:27:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.002+0000 7efd5ed9d640 1 --2- 172.21.15.45:0/430058932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd60065b90 0x7efd600a9db0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd500097a0 tx=0x7efd5002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6361b5e2ba532928 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:21.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 1 -- 172.21.15.45:0/430058932 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50031830 con 0x7efd60065b90 2023-07-29T13:27:21.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:27:21.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 10 monclient: dump: 2023-07-29T13:27:21.010 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.012 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 10 monclient: _finish_auth 0 2023-07-29T13:27:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 10 monclient: _check_auth_tickets 2023-07-29T13:27:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 1 -- 172.21.15.45:0/430058932 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd50031990 con 0x7efd60065b90 2023-07-29T13:27:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 1 -- 172.21.15.45:0/430058932 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50031d20 con 0x7efd60065b90 2023-07-29T13:27:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd65206640 5 monclient: authenticate success, global_id 4126 2023-07-29T13:27:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd65206640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd65206640 4 set_mon_vals no callback set 2023-07-29T13:27:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.009+0000 7efd5e59c640 10 monclient: dump: 2023-07-29T13:27:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.015 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.015 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:21.011+0000 7f9c90e32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-29T13:27:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.010+0000 7efd65206640 10 monclient: shutdown 2023-07-29T13:27:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.010+0000 7efd65206640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.010+0000 7efd65206640 1 -- 172.21.15.45:0/430058932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd60065b90 msgr2=0x7efd600a9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.010+0000 7efd65206640 1 --2- 172.21.15.45:0/430058932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd60065b90 0x7efd600a9db0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd500097a0 tx=0x7efd5002b8d0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.010+0000 7efd65206640 1 -- 172.21.15.45:0/430058932 shutdown_connections 2023-07-29T13:27:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.010+0000 7efd65206640 1 --2- 172.21.15.45:0/430058932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd60065b90 0x7efd600a9db0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.010+0000 7efd65206640 1 -- 172.21.15.45:0/430058932 >> 172.21.15.45:0/430058932 conn(0x7efd600b3b80 msgr2=0x7efd600b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.010+0000 7efd65206640 1 -- 172.21.15.45:0/430058932 shutdown_connections 2023-07-29T13:27:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.010+0000 7efd65206640 1 -- 172.21.15.45:0/430058932 wait complete. 2023-07-29T13:27:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.011+0000 7efd65206640 10 monclient: build_initial_monmap 2023-07-29T13:27:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.011+0000 7efd65206640 10 monclient: monmap: 2023-07-29T13:27:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2023-07-29T13:27:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2023-07-29T13:27:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:21.020 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.011+0000 7efd65206640 1 librados: starting msgr at 2023-07-29T13:27:21.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.011+0000 7efd65206640 1 librados: starting objecter 2023-07-29T13:27:21.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.011+0000 7efd65206640 1 Processor -- start 2023-07-29T13:27:21.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.012+0000 7efd65206640 1 -- start start 2023-07-29T13:27:21.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.012+0000 7efd65206640 1 librados: setting wanted keys 2023-07-29T13:27:21.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.012+0000 7efd65206640 1 librados: calling monclient init 2023-07-29T13:27:21.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.012+0000 7efd65206640 10 monclient: init 2023-07-29T13:27:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.012+0000 7efd65206640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.012+0000 7efd65206640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:27:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.012+0000 7efd65206640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efd60065b90 0x7efd601d9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.012+0000 7efd65206640 10 monclient(hunting): picked mon.noname-c con 0x7efd60065b90 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.012+0000 7efd65206640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd600aa960 0x7efd601d95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.012+0000 7efd65206640 10 monclient(hunting): picked mon.noname-b con 0x7efd600aa960 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.012+0000 7efd65206640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd60140da0 0x7efd600ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.013+0000 7efd65206640 10 monclient(hunting): picked mon.noname-a con 0x7efd60140da0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.013+0000 7efd65206640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.013+0000 7efd65206640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd600ad400 con 0x7efd60140da0 2023-07-29T13:27:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.013+0000 7efd65206640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.013+0000 7efd65206640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd601d9b00 con 0x7efd600aa960 2023-07-29T13:27:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.013+0000 7efd65206640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.013+0000 7efd65206640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd600719a0 con 0x7efd60065b90 2023-07-29T13:27:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.013+0000 7efd65206640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.013+0000 7efd65206640 10 monclient(hunting): authenticate will time out at 1364.240148s 2023-07-29T13:27:21.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd57fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd600aa960 0x7efd601d95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5ed9d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efd60065b90 0x7efd601d9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd60140da0 0x7efd600ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5ed9d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efd60065b90 0x7efd601d9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.45:54408/0 (socket says 172.21.15.45:54408) 2023-07-29T13:27:21.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5ed9d640 1 -- 172.21.15.45:0/2858650365 learned_addr learned my addr 172.21.15.45:0/2858650365 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-07-29T13:27:21.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5ed9d640 10 monclient(hunting): get_auth_request con 0x7efd60065b90 auth_method 0 2023-07-29T13:27:21.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5ed9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5ed9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5f59e640 10 monclient(hunting): get_auth_request con 0x7efd60140da0 auth_method 0 2023-07-29T13:27:21.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5f59e640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd5f59e640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd57fff640 10 monclient(hunting): get_auth_request con 0x7efd600aa960 auth_method 0 2023-07-29T13:27:21.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd57fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd57fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.014+0000 7efd57fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.015+0000 7efd5f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.015+0000 7efd5f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.015+0000 7efd5f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.015+0000 7efd57fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.015+0000 7efd57fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.015+0000 7efd57fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.015+0000 7efd5ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.015+0000 7efd5ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.015+0000 7efd5ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.018+0000 7efd5f59e640 10 monclient(hunting): handle_auth_done global_id 4131 payload 824 2023-07-29T13:27:21.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.018+0000 7efd5f59e640 1 -- 172.21.15.45:0/2858650365 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efd60065b90 msgr2=0x7efd601d9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.018+0000 7efd5f59e640 1 --2- 172.21.15.45:0/2858650365 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efd60065b90 0x7efd601d9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.018+0000 7efd5f59e640 1 -- 172.21.15.45:0/2858650365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd600aa960 msgr2=0x7efd601d95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.018+0000 7efd5f59e640 1 --2- 172.21.15.45:0/2858650365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd600aa960 0x7efd601d95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.018+0000 7efd5f59e640 10 monclient: _finish_hunting 0 2023-07-29T13:27:21.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.018+0000 7efd5f59e640 1 monclient: found mon.noname-a 2023-07-29T13:27:21.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.018+0000 7efd5f59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:21.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.018+0000 7efd5f59e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.018+0000 7efd5f59e640 1 -- 172.21.15.45:0/2858650365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd50009400 con 0x7efd60140da0 2023-07-29T13:27:21.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.020+0000 7efd5f59e640 1 --2- 172.21.15.45:0/2858650365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd60140da0 0x7efd600ac0c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efd5800caa0 tx=0x7efd5800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:21.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.021+0000 7efd577fe640 10 client.?.objecter ms_handle_connect 0x7efd60140da0 2023-07-29T13:27:21.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.021+0000 7efd577fe640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:21.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.024+0000 7efd577fe640 1 -- 172.21.15.45:0/2858650365 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5800ab40 con 0x7efd60140da0 2023-07-29T13:27:21.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.024+0000 7efd577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.024+0000 7efd577fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:27:21.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.024+0000 7efd577fe640 10 monclient: dump: 2023-07-29T13:27:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.032 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.032 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd577fe640 10 monclient: _finish_auth 0 2023-07-29T13:27:21.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd577fe640 10 monclient: _check_auth_tickets 2023-07-29T13:27:21.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:21.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd577fe640 1 -- 172.21.15.45:0/2858650365 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5800aca0 con 0x7efd60140da0 2023-07-29T13:27:21.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 5 monclient: authenticate success, global_id 4131 2023-07-29T13:27:21.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd577fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:21.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 10 monclient: _renew_subs 2023-07-29T13:27:21.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd577fe640 1 -- 172.21.15.45:0/2858650365 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58012600 con 0x7efd60140da0 2023-07-29T13:27:21.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 1 -- 172.21.15.45:0/2858650365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd60071c80 con 0x7efd60140da0 2023-07-29T13:27:21.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd577fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:27:21.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd5dd9b640 4 set_mon_vals no callback set 2023-07-29T13:27:21.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd577fe640 10 monclient: dump: 2023-07-29T13:27:21.034 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.034 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.034 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.034 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.034 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.034 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.034 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.035 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 10 client.4131.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 10 monclient: _renew_subs 2023-07-29T13:27:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 1 -- 172.21.15.45:0/2858650365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd600ac7e0 con 0x7efd60140da0 2023-07-29T13:27:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 1 librados: init done 2023-07-29T13:27:21.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:21.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:21.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.025+0000 7efd65206640 1 -- 172.21.15.45:0/2858650365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd600ae1e0 con 0x7efd60140da0 2023-07-29T13:27:21.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.144+0000 7efd577fe640 1 -- 172.21.15.45:0/2858650365 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efd58012a60 con 0x7efd60140da0 2023-07-29T13:27:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.150+0000 7efd577fe640 1 -- 172.21.15.45:0/2858650365 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7efd58007500 con 0x7efd60140da0 2023-07-29T13:27:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.150+0000 7efd577fe640 10 client.4131.objecter ms_dispatch 0x7efd60000b90 osd_map(5..5 src has 1..5) v4 2023-07-29T13:27:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.150+0000 7efd577fe640 3 client.4131.objecter handle_osd_map got epochs [5,5] > 0 2023-07-29T13:27:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.150+0000 7efd577fe640 3 client.4131.objecter handle_osd_map decoding full epoch 5 2023-07-29T13:27:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.150+0000 7efd577fe640 20 client.4131.objecter dump_active .. 0 homeless 2023-07-29T13:27:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.150+0000 7efd577fe640 1 -- 172.21.15.45:0/2858650365 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7efd58021060 con 0x7efd60140da0 2023-07-29T13:27:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.150+0000 7efd577fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.150+0000 7efd577fe640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.225+0000 7efd65206640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "e7224640-7d33-4bb7-81a5-99fc26da7763", "id": 3}] 2023-07-29T13:27:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.225+0000 7efd65206640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "e7224640-7d33-4bb7-81a5-99fc26da7763", "id": 3}] 2023-07-29T13:27:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.225+0000 7efd65206640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.225+0000 7efd65206640 1 -- 172.21.15.45:0/2858650365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e7224640-7d33-4bb7-81a5-99fc26da7763", "id": 3} v 0) v1 -- 0x7efd60067860 con 0x7efd60140da0 2023-07-29T13:27:21.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.280+0000 7efd577fe640 1 -- 172.21.15.45:0/2858650365 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e7224640-7d33-4bb7-81a5-99fc26da7763", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7efd58017070 con 0x7efd60140da0 2023-07-29T13:27:21.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.280+0000 7efd577fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "e7224640-7d33-4bb7-81a5-99fc26da7763", "id": 3}] 2023-07-29T13:27:21.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.280+0000 7efd577fe640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:21.283 INFO:teuthology.orchestra.run.smithi045.stdout:3 2023-07-29T13:27:21.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 10 librados: watch_flush enter 2023-07-29T13:27:21.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 10 librados: watch_flush exit 2023-07-29T13:27:21.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 20 client.4131.objecter shutdown clearing up homeless session... 2023-07-29T13:27:21.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 10 client.4131.objecter successfully canceled tick 2023-07-29T13:27:21.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 10 monclient: shutdown 2023-07-29T13:27:21.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:21.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 1 -- 172.21.15.45:0/2858650365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd60140da0 msgr2=0x7efd600ac0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 1 --2- 172.21.15.45:0/2858650365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd60140da0 0x7efd600ac0c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efd5800caa0 tx=0x7efd5800cea0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 1 -- 172.21.15.45:0/2858650365 shutdown_connections 2023-07-29T13:27:21.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 1 --2- 172.21.15.45:0/2858650365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd600aa960 0x7efd601d95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 1 --2- 172.21.15.45:0/2858650365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd60140da0 0x7efd600ac0c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 1 --2- 172.21.15.45:0/2858650365 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efd60065b90 0x7efd601d9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 1 -- 172.21.15.45:0/2858650365 >> 172.21.15.45:0/2858650365 conn(0x7efd600b3b80 msgr2=0x7efd6013deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:21.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 1 -- 172.21.15.45:0/2858650365 shutdown_connections 2023-07-29T13:27:21.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.281+0000 7efd65206640 1 -- 172.21.15.45:0/2858650365 wait complete. 2023-07-29T13:27:21.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.282+0000 7efd65206640 1 librados: shutdown 2023-07-29T13:27:21.293 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new 23cd76f6-788b-48cf-9438-e931659d3a1d 4 2023-07-29T13:27:21.375 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:21.374+0000 7ff142a2c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-29T13:27:21.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.397+0000 7f562387c640 10 monclient: get_monmap_and_config 2023-07-29T13:27:21.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.397+0000 7f562387c640 10 monclient: build_initial_monmap 2023-07-29T13:27:21.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.397+0000 7f562387c640 10 monclient: monmap: 2023-07-29T13:27:21.399 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:21.399 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.399 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:27:21.398172+0000 2023-07-29T13:27:21.399 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:27:21.398172+0000 2023-07-29T13:27:21.399 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:21.400 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.400 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:21.400 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:21.400 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:21.400 INFO:teuthology.orchestra.run.smithi045.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:21.400 INFO:teuthology.orchestra.run.smithi045.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:21.400 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 1 Processor -- start 2023-07-29T13:27:21.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 1 -- start start 2023-07-29T13:27:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 10 monclient: init 2023-07-29T13:27:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2023-07-29T13:27:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f561c0b3ae0 0x7f561c13dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 10 monclient(hunting): picked mon.noname-c con 0x7f561c0b3ae0 addr v2:172.21.15.45:3300/0 2023-07-29T13:27:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c13e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 10 monclient(hunting): picked mon.noname-b con 0x7f561c0a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f561c140c10 addr v1:172.21.15.45:6789/0 2023-07-29T13:27:21.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561c0b3eb0 con 0x7f561c140c10 2023-07-29T13:27:21.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561c0b0f10 con 0x7f561c0b3ae0 2023-07-29T13:27:21.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0a9950 con 0x7f561c0a8910 2023-07-29T13:27:21.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:21.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.398+0000 7f562387c640 10 monclient(hunting): authenticate will time out at 1364.626098s 2023-07-29T13:27:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.399+0000 7f5621618640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f561c0b3ae0 0x7f561c13dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.399+0000 7f5620e17640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c13e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5620e17640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c13e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.45:54414/0 (socket says 172.21.15.45:54414) 2023-07-29T13:27:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5620e17640 1 -- 172.21.15.45:0/3446931321 learned_addr learned my addr 172.21.15.45:0/3446931321 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-07-29T13:27:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5620e17640 10 monclient(hunting): get_auth_request con 0x7f561c0a8910 auth_method 0 2023-07-29T13:27:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5620e17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5620e17640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5620e17640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5621618640 10 monclient(hunting): get_auth_request con 0x7f561c0b3ae0 auth_method 0 2023-07-29T13:27:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5621618640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5621618640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5621618640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5621618640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5621618640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5621618640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5620e17640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5620e17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.400+0000 7f5620e17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.401+0000 7f5620e17640 1 -- 172.21.15.45:0/3446931321 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 msgr2=0x7f561c13e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-29T13:27:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.401+0000 7f5620e17640 1 -- 172.21.15.45:0/3446931321 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 msgr2=0x7f561c13e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-29T13:27:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.401+0000 7f5620e17640 1 --2- 172.21.15.45:0/3446931321 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c13e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-29T13:27:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.401+0000 7f5620e17640 1 --2- 172.21.15.45:0/3446931321 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c13e1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-29T13:27:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.402+0000 7f5621618640 10 monclient(hunting): handle_auth_done global_id 4132 payload 274 2023-07-29T13:27:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.402+0000 7f5621618640 1 -- 172.21.15.45:0/3446931321 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 msgr2=0x7f561c13e1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-29T13:27:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.402+0000 7f5621618640 1 --2- 172.21.15.45:0/3446931321 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c13e1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.402+0000 7f5621618640 1 -- 172.21.15.45:0/3446931321 >> v1:172.21.15.45:6789/0 conn(0x7f561c140c10 legacy=0x7f561c141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.402+0000 7f5621618640 10 monclient: _finish_hunting 0 2023-07-29T13:27:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.402+0000 7f5621618640 1 monclient: found mon.noname-c 2023-07-29T13:27:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.402+0000 7f5621618640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.402+0000 7f5621618640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:27:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.402+0000 7f5621618640 1 -- 172.21.15.45:0/3446931321 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c0a9c30 con 0x7f561c0b3ae0 2023-07-29T13:27:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.404+0000 7f5621618640 1 --2- 172.21.15.45:0/3446931321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f561c0b3ae0 0x7f561c13dc90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5618009d90 tx=0x7f561802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1afd092bc8d6880e server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 1 -- 172.21.15.45:0/3446931321 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561802cbc0 con 0x7f561c0b3ae0 2023-07-29T13:27:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:27:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 10 monclient: dump: 2023-07-29T13:27:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.414 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 10 monclient: _finish_auth 0 2023-07-29T13:27:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 10 monclient: _check_auth_tickets 2023-07-29T13:27:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 5 monclient: authenticate success, global_id 4132 2023-07-29T13:27:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 1 -- 172.21.15.45:0/3446931321 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561802cd20 con 0x7f561c0b3ae0 2023-07-29T13:27:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:27:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 4 set_mon_vals no callback set 2023-07-29T13:27:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 1 -- 172.21.15.45:0/3446931321 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5618034660 con 0x7f561c0b3ae0 2023-07-29T13:27:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f5613fff640 10 monclient: dump: 2023-07-29T13:27:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.417 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 10 monclient: shutdown 2023-07-29T13:27:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 1 -- 172.21.15.45:0/3446931321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f561c0b3ae0 msgr2=0x7f561c13dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 1 --2- 172.21.15.45:0/3446931321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f561c0b3ae0 0x7f561c13dc90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5618009d90 tx=0x7f561802be10 comp rx=0 tx=0).stop 2023-07-29T13:27:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 1 -- 172.21.15.45:0/3446931321 shutdown_connections 2023-07-29T13:27:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 1 --2- 172.21.15.45:0/3446931321 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c13e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 1 --2- 172.21.15.45:0/3446931321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f561c0b3ae0 0x7f561c13dc90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 1 -- 172.21.15.45:0/3446931321 >> 172.21.15.45:0/3446931321 conn(0x7f561c070fc0 msgr2=0x7f561c0713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 1 -- 172.21.15.45:0/3446931321 shutdown_connections 2023-07-29T13:27:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.411+0000 7f562387c640 1 -- 172.21.15.45:0/3446931321 wait complete. 2023-07-29T13:27:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.412+0000 7f562387c640 10 monclient: build_initial_monmap 2023-07-29T13:27:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.412+0000 7f562387c640 10 monclient: monmap: 2023-07-29T13:27:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2023-07-29T13:27:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2023-07-29T13:27:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.412+0000 7f562387c640 1 librados: starting msgr at 2023-07-29T13:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.412+0000 7f562387c640 1 librados: starting objecter 2023-07-29T13:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.412+0000 7f562387c640 1 Processor -- start 2023-07-29T13:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 1 -- start start 2023-07-29T13:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 1 librados: setting wanted keys 2023-07-29T13:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 1 librados: calling monclient init 2023-07-29T13:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 10 monclient: init 2023-07-29T13:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c0ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 10 monclient(hunting): picked mon.noname-c con 0x7f561c0a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f561c0b3ae0 0x7f561c0acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 10 monclient(hunting): picked mon.noname-b con 0x7f561c0b3ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561c140c10 0x7f561c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 10 monclient(hunting): picked mon.noname-a con 0x7f561c140c10 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0adf30 con 0x7f561c140c10 2023-07-29T13:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0af560 con 0x7f561c0b3ae0 2023-07-29T13:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0ad0f0 con 0x7f561c0a8910 2023-07-29T13:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.413+0000 7f562387c640 10 monclient(hunting): authenticate will time out at 1364.640684s 2023-07-29T13:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621e19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561c140c10 0x7f561c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621618640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c0ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621e19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561c140c10 0x7f561c0afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.45:47000/0 (socket says 172.21.15.45:47000) 2023-07-29T13:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621e19640 1 -- 172.21.15.45:0/2312191264 learned_addr learned my addr 172.21.15.45:0/2312191264 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-07-29T13:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621e19640 10 monclient(hunting): get_auth_request con 0x7f561c140c10 auth_method 0 2023-07-29T13:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621e19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621e19640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621e19640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621618640 10 monclient(hunting): get_auth_request con 0x7f561c0a8910 auth_method 0 2023-07-29T13:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621618640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621618640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.414+0000 7f5621618640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.415+0000 7f5621e19640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.415+0000 7f5621e19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.415+0000 7f5621e19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.415+0000 7f5621618640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.415+0000 7f5621618640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.415+0000 7f5621618640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.415+0000 7f5620e17640 1 --2- 172.21.15.45:0/2312191264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f561c0b3ae0 0x7f561c0acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.415+0000 7f5620e17640 10 monclient(hunting): get_auth_request con 0x7f561c0b3ae0 auth_method 0 2023-07-29T13:27:21.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.415+0000 7f5620e17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.415+0000 7f5620e17640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.415+0000 7f5620e17640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.416+0000 7f5620e17640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.416+0000 7f5620e17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.416+0000 7f5620e17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.417+0000 7f5621e19640 10 monclient(hunting): handle_auth_done global_id 4134 payload 824 2023-07-29T13:27:21.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.418+0000 7f5621e19640 1 -- 172.21.15.45:0/2312191264 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 msgr2=0x7f561c0ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.418+0000 7f5621e19640 1 --2- 172.21.15.45:0/2312191264 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c0ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.418+0000 7f5621e19640 1 -- 172.21.15.45:0/2312191264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f561c0b3ae0 msgr2=0x7f561c0acbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.418+0000 7f5621e19640 1 --2- 172.21.15.45:0/2312191264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f561c0b3ae0 0x7f561c0acbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.418+0000 7f5621e19640 10 monclient: _finish_hunting 0 2023-07-29T13:27:21.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.418+0000 7f5621e19640 1 monclient: found mon.noname-a 2023-07-29T13:27:21.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.418+0000 7f5621e19640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:21.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.418+0000 7f5621e19640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.418+0000 7f5621e19640 1 -- 172.21.15.45:0/2312191264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5618009a70 con 0x7f561c140c10 2023-07-29T13:27:21.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.418+0000 7f5621618640 1 --2- 172.21.15.45:0/2312191264 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c0ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-29T13:27:21.432 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:21.422+0000 7f9c90e32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-29T13:27:21.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.420+0000 7f5621e19640 1 --2- 172.21.15.45:0/2312191264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561c140c10 0x7f561c0afa80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f561400b410 tx=0x7f561400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:21.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.420+0000 7f5611ffb640 10 client.?.objecter ms_handle_connect 0x7f561c140c10 2023-07-29T13:27:21.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.420+0000 7f5611ffb640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:21.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 1 -- 172.21.15.45:0/2312191264 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614004020 con 0x7f561c140c10 2023-07-29T13:27:21.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:27:21.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 10 monclient: dump: 2023-07-29T13:27:21.434 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.435 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.435 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.435 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.435 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.435 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.435 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.435 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.435 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.436 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 10 monclient: _finish_auth 0 2023-07-29T13:27:21.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 10 monclient: _check_auth_tickets 2023-07-29T13:27:21.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:21.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 1 -- 172.21.15.45:0/2312191264 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5614004180 con 0x7f561c140c10 2023-07-29T13:27:21.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:21.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 1 -- 172.21.15.45:0/2312191264 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614011630 con 0x7f561c140c10 2023-07-29T13:27:21.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:27:21.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f5611ffb640 10 monclient: dump: 2023-07-29T13:27:21.437 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.437 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.437 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.437 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.437 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.438 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.438 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.438 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.438 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.438 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f562387c640 5 monclient: authenticate success, global_id 4134 2023-07-29T13:27:21.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f562387c640 10 monclient: _renew_subs 2023-07-29T13:27:21.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f562387c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f562387c640 1 -- 172.21.15.45:0/2312191264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561c0abd00 con 0x7f561c140c10 2023-07-29T13:27:21.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.423+0000 7f56137fe640 4 set_mon_vals no callback set 2023-07-29T13:27:21.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.424+0000 7f562387c640 10 client.4134.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:21.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.424+0000 7f562387c640 10 monclient: _renew_subs 2023-07-29T13:27:21.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.424+0000 7f562387c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.424+0000 7f562387c640 1 -- 172.21.15.45:0/2312191264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561c0b01a0 con 0x7f561c140c10 2023-07-29T13:27:21.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.424+0000 7f562387c640 1 librados: init done 2023-07-29T13:27:21.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.425+0000 7f562387c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:21.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.425+0000 7f562387c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:21.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.425+0000 7f562387c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.425+0000 7f562387c640 1 -- 172.21.15.45:0/2312191264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561c0b5b80 con 0x7f561c140c10 2023-07-29T13:27:21.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.426+0000 7f5611ffb640 1 -- 172.21.15.45:0/2312191264 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5614011a90 con 0x7f561c140c10 2023-07-29T13:27:21.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.529+0000 7f5611ffb640 1 -- 172.21.15.45:0/2312191264 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f5614011d40 con 0x7f561c140c10 2023-07-29T13:27:21.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.529+0000 7f5611ffb640 10 client.4134.objecter ms_dispatch 0x7f561c000b90 osd_map(6..6 src has 1..6) v4 2023-07-29T13:27:21.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.529+0000 7f5611ffb640 3 client.4134.objecter handle_osd_map got epochs [6,6] > 0 2023-07-29T13:27:21.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.529+0000 7f5611ffb640 3 client.4134.objecter handle_osd_map decoding full epoch 6 2023-07-29T13:27:21.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.529+0000 7f5611ffb640 20 client.4134.objecter dump_active .. 0 homeless 2023-07-29T13:27:21.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.529+0000 7f5611ffb640 1 -- 172.21.15.45:0/2312191264 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f5614015030 con 0x7f561c140c10 2023-07-29T13:27:21.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.529+0000 7f5611ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.529+0000 7f5611ffb640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:21.581 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:21.580+0000 7ff142a2c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-29T13:27:21.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.602+0000 7f562387c640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "23cd76f6-788b-48cf-9438-e931659d3a1d", "id": 4}] 2023-07-29T13:27:21.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.602+0000 7f562387c640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "23cd76f6-788b-48cf-9438-e931659d3a1d", "id": 4}] 2023-07-29T13:27:21.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.602+0000 7f562387c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.602+0000 7f562387c640 1 -- 172.21.15.45:0/2312191264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd new", "uuid": "23cd76f6-788b-48cf-9438-e931659d3a1d", "id": 4} v 0) v1 -- 0x7f561c0b58a0 con 0x7f561c140c10 2023-07-29T13:27:21.628 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:21.627+0000 7f9c90e32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-29T13:27:21.639 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:21.638+0000 7ff142a2c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-29T13:27:21.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.658+0000 7f5611ffb640 1 -- 172.21.15.45:0/2312191264 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "23cd76f6-788b-48cf-9438-e931659d3a1d", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5614003d00 con 0x7f561c140c10 2023-07-29T13:27:21.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.658+0000 7f5611ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "23cd76f6-788b-48cf-9438-e931659d3a1d", "id": 4}] 2023-07-29T13:27:21.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.658+0000 7f5611ffb640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:21.660 INFO:teuthology.orchestra.run.smithi045.stdout:4 2023-07-29T13:27:21.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 10 librados: watch_flush enter 2023-07-29T13:27:21.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 10 librados: watch_flush exit 2023-07-29T13:27:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 20 client.4134.objecter shutdown clearing up homeless session... 2023-07-29T13:27:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 10 client.4134.objecter successfully canceled tick 2023-07-29T13:27:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 10 monclient: shutdown 2023-07-29T13:27:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 1 -- 172.21.15.45:0/2312191264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561c140c10 msgr2=0x7f561c0afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 1 --2- 172.21.15.45:0/2312191264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561c140c10 0x7f561c0afa80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f561400b410 tx=0x7f561400b8d0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 1 -- 172.21.15.45:0/2312191264 shutdown_connections 2023-07-29T13:27:21.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 1 --2- 172.21.15.45:0/2312191264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f561c0b3ae0 0x7f561c0acbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 1 --2- 172.21.15.45:0/2312191264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561c140c10 0x7f561c0afa80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 1 --2- 172.21.15.45:0/2312191264 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f561c0a8910 0x7f561c0ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 1 -- 172.21.15.45:0/2312191264 >> 172.21.15.45:0/2312191264 conn(0x7f561c070fc0 msgr2=0x7f561c13dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:21.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 1 -- 172.21.15.45:0/2312191264 shutdown_connections 2023-07-29T13:27:21.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 1 -- 172.21.15.45:0/2312191264 wait complete. 2023-07-29T13:27:21.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.659+0000 7f562387c640 1 librados: shutdown 2023-07-29T13:27:21.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new 0ddb1e3f-0f65-40d6-ab50-5dc521774445 5 2023-07-29T13:27:21.690 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:21.689+0000 7f9c90e32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-29T13:27:21.701 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:21.700+0000 7ff142a2c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-29T13:27:21.743 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:21.741+0000 7f9c90e32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-29T13:27:21.771 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:21.769+0000 7ff142a2c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-29T13:27:21.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.776+0000 7f91057e3640 10 monclient: get_monmap_and_config 2023-07-29T13:27:21.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.776+0000 7f91057e3640 10 monclient: build_initial_monmap 2023-07-29T13:27:21.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.776+0000 7f91057e3640 10 monclient: monmap: 2023-07-29T13:27:21.777 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:27:21.776756+0000 2023-07-29T13:27:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:27:21.776756+0000 2023-07-29T13:27:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:21.779 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.776+0000 7f91057e3640 1 Processor -- start 2023-07-29T13:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.776+0000 7f91057e3640 1 -- start start 2023-07-29T13:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.776+0000 7f91057e3640 10 monclient: init 2023-07-29T13:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 10 monclient: _add_conns ranks=[4,3,2,1,0] 2023-07-29T13:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91000b3ae0 0x7f910013dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 10 monclient(hunting): picked mon.noname-c con 0x7f91000b3ae0 addr v2:172.21.15.45:3300/0 2023-07-29T13:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91000a8910 0x7f910013e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 10 monclient(hunting): picked mon.noname-a con 0x7f91000a8910 addr v2:172.21.15.40:3300/0 2023-07-29T13:27:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9100140c10 0x7f9100141070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 10 monclient(hunting): picked mon.noname-b con 0x7f9100140c10 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91000b0f10 con 0x7f91000a8910 2023-07-29T13:27:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91000b3eb0 con 0x7f91000b3ae0 2023-07-29T13:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91000a9950 con 0x7f9100140c10 2023-07-29T13:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.777+0000 7f91057e3640 10 monclient(hunting): authenticate will time out at 1365.004572s 2023-07-29T13:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90f7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91000a8910 0x7f910013e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90feffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91000b3ae0 0x7f910013dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90ff7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9100140c10 0x7f9100141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90f7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91000a8910 0x7f910013e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.45:47004/0 (socket says 172.21.15.45:47004) 2023-07-29T13:27:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90f7fff640 1 -- 172.21.15.45:0/364822903 learned_addr learned my addr 172.21.15.45:0/364822903 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-07-29T13:27:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90f7fff640 10 monclient(hunting): get_auth_request con 0x7f91000a8910 auth_method 0 2023-07-29T13:27:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90f7fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90f7fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90feffd640 10 monclient(hunting): get_auth_request con 0x7f91000b3ae0 auth_method 0 2023-07-29T13:27:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90feffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90feffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90ff7fe640 10 monclient(hunting): get_auth_request con 0x7f9100140c10 auth_method 0 2023-07-29T13:27:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90ff7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90ff7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.778+0000 7f90f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.779+0000 7f90feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.779+0000 7f90feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.779+0000 7f90feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.779+0000 7f90ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.779+0000 7f90ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.779+0000 7f90ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.780+0000 7f90f7fff640 10 monclient(hunting): handle_auth_done global_id 4137 payload 274 2023-07-29T13:27:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.780+0000 7f90f7fff640 1 -- 172.21.15.45:0/364822903 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9100140c10 msgr2=0x7f9100141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.780+0000 7f90f7fff640 1 --2- 172.21.15.45:0/364822903 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9100140c10 0x7f9100141070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.780+0000 7f90f7fff640 1 -- 172.21.15.45:0/364822903 >> v2:172.21.15.45:3300/0 conn(0x7f91000b3ae0 msgr2=0x7f910013dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.780+0000 7f90f7fff640 1 --2- 172.21.15.45:0/364822903 >> v2:172.21.15.45:3300/0 conn(0x7f91000b3ae0 0x7f910013dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.780+0000 7f90f7fff640 10 monclient: _finish_hunting 0 2023-07-29T13:27:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.780+0000 7f90f7fff640 1 monclient: found mon.noname-a 2023-07-29T13:27:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.780+0000 7f90f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.780+0000 7f90f7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.780+0000 7f90f7fff640 1 -- 172.21.15.45:0/364822903 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91000a9c30 con 0x7f91000a8910 2023-07-29T13:27:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.780+0000 7f90feffd640 1 --2- 172.21.15.45:0/364822903 >> v2:172.21.15.45:3300/0 conn(0x7f91000b3ae0 0x7f910013dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-29T13:27:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.782+0000 7f90f7fff640 1 --2- 172.21.15.45:0/364822903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91000a8910 0x7f910013e1d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9100066830 tx=0x7f90e802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ed98eddff31ab7 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 1 -- 172.21.15.45:0/364822903 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e80317c0 con 0x7f91000a8910 2023-07-29T13:27:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:27:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 10 monclient: dump: 2023-07-29T13:27:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.799 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 10 monclient: _finish_auth 0 2023-07-29T13:27:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 10 monclient: _check_auth_tickets 2023-07-29T13:27:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 1 -- 172.21.15.45:0/364822903 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e8031920 con 0x7f91000a8910 2023-07-29T13:27:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 5 monclient: authenticate success, global_id 4137 2023-07-29T13:27:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:27:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 4 set_mon_vals no callback set 2023-07-29T13:27:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 1 -- 172.21.15.45:0/364822903 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8031cb0 con 0x7f91000a8910 2023-07-29T13:27:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:27:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f90fe7fc640 10 monclient: dump: 2023-07-29T13:27:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.802 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 10 monclient: shutdown 2023-07-29T13:27:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 1 -- 172.21.15.45:0/364822903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91000a8910 msgr2=0x7f910013e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 1 --2- 172.21.15.45:0/364822903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91000a8910 0x7f910013e1d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9100066830 tx=0x7f90e802b860 comp rx=0 tx=0).stop 2023-07-29T13:27:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 1 -- 172.21.15.45:0/364822903 shutdown_connections 2023-07-29T13:27:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 1 --2- 172.21.15.45:0/364822903 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9100140c10 0x7f9100141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 1 --2- 172.21.15.45:0/364822903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91000a8910 0x7f910013e1d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 1 --2- 172.21.15.45:0/364822903 >> v2:172.21.15.45:3300/0 conn(0x7f91000b3ae0 0x7f910013dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 1 -- 172.21.15.45:0/364822903 >> 172.21.15.45:0/364822903 conn(0x7f9100070fc0 msgr2=0x7f91000713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 1 -- 172.21.15.45:0/364822903 shutdown_connections 2023-07-29T13:27:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.786+0000 7f91057e3640 1 -- 172.21.15.45:0/364822903 wait complete. 2023-07-29T13:27:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.787+0000 7f91057e3640 10 monclient: build_initial_monmap 2023-07-29T13:27:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.787+0000 7f91057e3640 10 monclient: monmap: 2023-07-29T13:27:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2023-07-29T13:27:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2023-07-29T13:27:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:21.805 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.787+0000 7f91057e3640 1 librados: starting msgr at 2023-07-29T13:27:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.787+0000 7f91057e3640 1 librados: starting objecter 2023-07-29T13:27:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.787+0000 7f91057e3640 1 Processor -- start 2023-07-29T13:27:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.787+0000 7f91057e3640 1 -- start start 2023-07-29T13:27:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.787+0000 7f91057e3640 1 librados: setting wanted keys 2023-07-29T13:27:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.787+0000 7f91057e3640 1 librados: calling monclient init 2023-07-29T13:27:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.787+0000 7f91057e3640 10 monclient: init 2023-07-29T13:27:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 10 monclient: _add_conns ranks=[1,0,2] 2023-07-29T13:27:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000a8910 0x7f91000b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 10 monclient(hunting): picked mon.noname-b con 0x7f91000a8910 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91000b3ae0 0x7f91001438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 10 monclient(hunting): picked mon.noname-a con 0x7f91000b3ae0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9100140c10 0x7f91001d98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 10 monclient(hunting): picked mon.noname-c con 0x7f9100140c10 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91001d9130 con 0x7f91000b3ae0 2023-07-29T13:27:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91000aae00 con 0x7f91000a8910 2023-07-29T13:27:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91001d9380 con 0x7f9100140c10 2023-07-29T13:27:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.788+0000 7f91057e3640 10 monclient(hunting): authenticate will time out at 1365.015655s 2023-07-29T13:27:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91000b3ae0 0x7f91001438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90feffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000a8910 0x7f91000b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90ff7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9100140c10 0x7f91001d98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91000b3ae0 0x7f91001438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.45:47016/0 (socket says 172.21.15.45:47016) 2023-07-29T13:27:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90f7fff640 1 -- 172.21.15.45:0/1308653203 learned_addr learned my addr 172.21.15.45:0/1308653203 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-07-29T13:27:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90f7fff640 10 monclient(hunting): get_auth_request con 0x7f91000b3ae0 auth_method 0 2023-07-29T13:27:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90f7fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90f7fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90feffd640 10 monclient(hunting): get_auth_request con 0x7f91000a8910 auth_method 0 2023-07-29T13:27:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90feffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90feffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90ff7fe640 10 monclient(hunting): get_auth_request con 0x7f9100140c10 auth_method 0 2023-07-29T13:27:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90ff7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90ff7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.789+0000 7f90f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.790+0000 7f90ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.790+0000 7f90ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.790+0000 7f90ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.790+0000 7f90feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:21.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.790+0000 7f90feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:21.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.790+0000 7f90feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:21.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.792+0000 7f90f7fff640 10 monclient(hunting): handle_auth_done global_id 4140 payload 824 2023-07-29T13:27:21.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.792+0000 7f90f7fff640 1 -- 172.21.15.45:0/1308653203 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9100140c10 msgr2=0x7f91001d98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.792+0000 7f90f7fff640 1 --2- 172.21.15.45:0/1308653203 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9100140c10 0x7f91001d98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.792+0000 7f90f7fff640 1 -- 172.21.15.45:0/1308653203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000a8910 msgr2=0x7f91000b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:21.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.792+0000 7f90f7fff640 1 --2- 172.21.15.45:0/1308653203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000a8910 0x7f91000b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:21.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.792+0000 7f90f7fff640 10 monclient: _finish_hunting 0 2023-07-29T13:27:21.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.792+0000 7f90f7fff640 1 monclient: found mon.noname-a 2023-07-29T13:27:21.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.792+0000 7f90f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:21.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.792+0000 7f90f7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.792+0000 7f90f7fff640 1 -- 172.21.15.45:0/1308653203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f0009400 con 0x7f91000b3ae0 2023-07-29T13:27:21.814 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:21.798+0000 7f9c90e32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-29T13:27:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.795+0000 7f90f7fff640 1 --2- 172.21.15.45:0/1308653203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91000b3ae0 0x7f91001438c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90e802b8a0 tx=0x7f90e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.795+0000 7f90f77fe640 10 client.?.objecter ms_handle_connect 0x7f91000b3ae0 2023-07-29T13:27:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.795+0000 7f90f77fe640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 1 -- 172.21.15.45:0/1308653203 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8032860 con 0x7f91000b3ae0 2023-07-29T13:27:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:27:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 10 monclient: dump: 2023-07-29T13:27:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.818 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 10 monclient: _finish_auth 0 2023-07-29T13:27:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 10 monclient: _check_auth_tickets 2023-07-29T13:27:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f91057e3640 5 monclient: authenticate success, global_id 4140 2023-07-29T13:27:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 1 -- 172.21.15.45:0/1308653203 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e80329e0 con 0x7f91000b3ae0 2023-07-29T13:27:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 1 -- 172.21.15.45:0/1308653203 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8032d90 con 0x7f91000b3ae0 2023-07-29T13:27:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:27:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90fdffb640 4 set_mon_vals no callback set 2023-07-29T13:27:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f90f77fe640 10 monclient: dump: 2023-07-29T13:27:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:21.821 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f91057e3640 10 monclient: _renew_subs 2023-07-29T13:27:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f91057e3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f91057e3640 1 -- 172.21.15.45:0/1308653203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e80094d0 con 0x7f91000b3ae0 2023-07-29T13:27:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f91057e3640 10 client.4140.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f91057e3640 10 monclient: _renew_subs 2023-07-29T13:27:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f91057e3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f91057e3640 1 -- 172.21.15.45:0/1308653203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91001d9de0 con 0x7f91000b3ae0 2023-07-29T13:27:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.798+0000 7f91057e3640 1 librados: init done 2023-07-29T13:27:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.799+0000 7f91057e3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.799+0000 7f91057e3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.799+0000 7f91057e3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.799+0000 7f91057e3640 1 -- 172.21.15.45:0/1308653203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91001da090 con 0x7f91000b3ae0 2023-07-29T13:27:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.899+0000 7f90f77fe640 1 -- 172.21.15.45:0/1308653203 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f90e804e750 con 0x7f91000b3ae0 2023-07-29T13:27:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.903+0000 7f90f77fe640 1 -- 172.21.15.45:0/1308653203 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f90e803b970 con 0x7f91000b3ae0 2023-07-29T13:27:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.903+0000 7f90f77fe640 10 client.4140.objecter ms_dispatch 0x7f9100000b90 osd_map(7..7 src has 1..7) v4 2023-07-29T13:27:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.903+0000 7f90f77fe640 3 client.4140.objecter handle_osd_map got epochs [7,7] > 0 2023-07-29T13:27:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.903+0000 7f90f77fe640 3 client.4140.objecter handle_osd_map decoding full epoch 7 2023-07-29T13:27:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.903+0000 7f90f77fe640 20 client.4140.objecter dump_active .. 0 homeless 2023-07-29T13:27:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.903+0000 7f90f77fe640 1 -- 172.21.15.45:0/1308653203 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f90e804ebe0 con 0x7f91000b3ae0 2023-07-29T13:27:21.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.903+0000 7f90f77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:21.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.903+0000 7f90f77fe640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.976+0000 7f91057e3640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "0ddb1e3f-0f65-40d6-ab50-5dc521774445", "id": 5}] 2023-07-29T13:27:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.976+0000 7f91057e3640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "0ddb1e3f-0f65-40d6-ab50-5dc521774445", "id": 5}] 2023-07-29T13:27:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.976+0000 7f91057e3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:21.976+0000 7f91057e3640 1 -- 172.21.15.45:0/1308653203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0ddb1e3f-0f65-40d6-ab50-5dc521774445", "id": 5} v 0) v1 -- 0x7f91001d9540 con 0x7f91000b3ae0 2023-07-29T13:27:22.025 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:22.023+0000 7ff142a2c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-29T13:27:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.042+0000 7f90f77fe640 1 -- 172.21.15.45:0/1308653203 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0ddb1e3f-0f65-40d6-ab50-5dc521774445", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f90e804ed80 con 0x7f91000b3ae0 2023-07-29T13:27:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.042+0000 7f90f77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "0ddb1e3f-0f65-40d6-ab50-5dc521774445", "id": 5}] 2023-07-29T13:27:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.042+0000 7f90f77fe640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:22.045 INFO:teuthology.orchestra.run.smithi045.stdout:5 2023-07-29T13:27:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 10 librados: watch_flush enter 2023-07-29T13:27:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 10 librados: watch_flush exit 2023-07-29T13:27:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 20 client.4140.objecter shutdown clearing up homeless session... 2023-07-29T13:27:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 10 client.4140.objecter successfully canceled tick 2023-07-29T13:27:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 10 monclient: shutdown 2023-07-29T13:27:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 1 -- 172.21.15.45:0/1308653203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91000b3ae0 msgr2=0x7f91001438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 1 --2- 172.21.15.45:0/1308653203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91000b3ae0 0x7f91001438c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90e802b8a0 tx=0x7f90e80046c0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 1 -- 172.21.15.45:0/1308653203 shutdown_connections 2023-07-29T13:27:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 1 --2- 172.21.15.45:0/1308653203 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9100140c10 0x7f91001d98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 1 --2- 172.21.15.45:0/1308653203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91000b3ae0 0x7f91001438c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 1 --2- 172.21.15.45:0/1308653203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000a8910 0x7f91000b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 1 -- 172.21.15.45:0/1308653203 >> 172.21.15.45:0/1308653203 conn(0x7f9100070fc0 msgr2=0x7f91000b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 1 -- 172.21.15.45:0/1308653203 shutdown_connections 2023-07-29T13:27:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 1 -- 172.21.15.45:0/1308653203 wait complete. 2023-07-29T13:27:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.043+0000 7f91057e3640 1 librados: shutdown 2023-07-29T13:27:22.052 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:22.051+0000 7f9c90e32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-29T13:27:22.055 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new 4b9d2830-9f2c-438c-9c0b-b6692e21f921 6 2023-07-29T13:27:22.081 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:22.080+0000 7ff142a2c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-29T13:27:22.107 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:22.106+0000 7f9c90e32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-29T13:27:22.143 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:22.142+0000 7ff142a2c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-29T13:27:22.170 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:22.168+0000 7f9c90e32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-29T13:27:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.186+0000 7fe06c1e0640 10 monclient: get_monmap_and_config 2023-07-29T13:27:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.186+0000 7fe06c1e0640 10 monclient: build_initial_monmap 2023-07-29T13:27:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.186+0000 7fe06c1e0640 10 monclient: monmap: 2023-07-29T13:27:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:27:22.187073+0000 2023-07-29T13:27:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:27:22.187073+0000 2023-07-29T13:27:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:22.190 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.186+0000 7fe06c1e0640 1 Processor -- start 2023-07-29T13:27:22.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 1 -- start start 2023-07-29T13:27:22.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 10 monclient: init 2023-07-29T13:27:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2023-07-29T13:27:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe0640b3ae0 addr v1:172.21.15.40:6789/0 2023-07-29T13:27:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe06413e3a0 addr v1:172.21.15.45:6789/0 2023-07-29T13:27:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe0640a8910 0x7fe06414c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 10 monclient(hunting): picked mon.noname-c con 0x7fe0640a8910 addr v2:172.21.15.45:3300/0 2023-07-29T13:27:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0640b3eb0 con 0x7fe0640b3ae0 2023-07-29T13:27:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0640a9950 con 0x7fe06413e3a0 2023-07-29T13:27:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0640b0f10 con 0x7fe0640a8910 2023-07-29T13:27:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.187+0000 7fe06c1e0640 10 monclient(hunting): authenticate will time out at 1365.414943s 2023-07-29T13:27:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.188+0000 7fe069f7c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe0640b3ae0 0x7fe06413dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.45:37856/0 (socket says 172.21.15.45:37856) 2023-07-29T13:27:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.188+0000 7fe069f7c640 1 -- 172.21.15.45:0/510750124 learned_addr learned my addr 172.21.15.45:0/510750124 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-07-29T13:27:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.188+0000 7fe06a77d640 1 --2- 172.21.15.45:0/510750124 >> v2:172.21.15.45:3300/0 conn(0x7fe0640a8910 0x7fe06414c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:22.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.188+0000 7fe06a77d640 10 monclient(hunting): get_auth_request con 0x7fe0640a8910 auth_method 0 2023-07-29T13:27:22.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.188+0000 7fe06a77d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:22.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.188+0000 7fe06a77d640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:22.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.188+0000 7fe06a77d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:22.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.189+0000 7fe06a77d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:22.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.189+0000 7fe06a77d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:22.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.189+0000 7fe06a77d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:22.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe068f7a640 1 -- 172.21.15.45:0/510750124 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211637644 0 0) 0x7fe0640b3eb0 con 0x7fe0640b3ae0 2023-07-29T13:27:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe068f7a640 10 monclient(hunting): _negotiate 2023-07-29T13:27:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe068f7a640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe068f7a640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe068f7a640 10 monclient(hunting): my global_id is 4143 2023-07-29T13:27:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe068f7a640 1 -- 172.21.15.45:0/510750124 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe050003520 con 0x7fe0640b3ae0 2023-07-29T13:27:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe06a77d640 10 monclient(hunting): handle_auth_done global_id 4144 payload 274 2023-07-29T13:27:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe06a77d640 1 -- 172.21.15.45:0/510750124 >> v1:172.21.15.45:6789/0 conn(0x7fe06413e3a0 legacy=0x7fe064140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe06a77d640 1 -- 172.21.15.45:0/510750124 >> v1:172.21.15.40:6789/0 conn(0x7fe0640b3ae0 legacy=0x7fe06413dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe06a77d640 10 monclient: _finish_hunting 0 2023-07-29T13:27:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe06a77d640 1 monclient: found mon.noname-c 2023-07-29T13:27:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe06a77d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe06a77d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:27:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.191+0000 7fe06a77d640 1 -- 172.21.15.45:0/510750124 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0640a9c30 con 0x7fe0640a8910 2023-07-29T13:27:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.193+0000 7fe06a77d640 1 --2- 172.21.15.45:0/510750124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0640a8910 0x7fe06414c530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009d90 tx=0x7fe05c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb05fa1e1ee4821 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:22.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 1 -- 172.21.15.45:0/510750124 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c02cc60 con 0x7fe0640a8910 2023-07-29T13:27:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:27:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 10 monclient: dump: 2023-07-29T13:27:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:22.202 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 10 monclient: _finish_auth 0 2023-07-29T13:27:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 10 monclient: _check_auth_tickets 2023-07-29T13:27:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 1 -- 172.21.15.45:0/510750124 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c02cdc0 con 0x7fe0640a8910 2023-07-29T13:27:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 1 -- 172.21.15.45:0/510750124 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c034660 con 0x7fe0640a8910 2023-07-29T13:27:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe068f7a640 10 monclient: dump: 2023-07-29T13:27:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:22.205 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe06c1e0640 5 monclient: authenticate success, global_id 4144 2023-07-29T13:27:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe06c1e0640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.199+0000 7fe06c1e0640 4 set_mon_vals no callback set 2023-07-29T13:27:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.200+0000 7fe06c1e0640 10 monclient: shutdown 2023-07-29T13:27:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.200+0000 7fe06c1e0640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.200+0000 7fe06c1e0640 1 -- 172.21.15.45:0/510750124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0640a8910 msgr2=0x7fe06414c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.200+0000 7fe06c1e0640 1 --2- 172.21.15.45:0/510750124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0640a8910 0x7fe06414c530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009d90 tx=0x7fe05c02beb0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.200+0000 7fe06c1e0640 1 -- 172.21.15.45:0/510750124 shutdown_connections 2023-07-29T13:27:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.200+0000 7fe06c1e0640 1 --2- 172.21.15.45:0/510750124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0640a8910 0x7fe06414c530 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.200+0000 7fe06c1e0640 1 -- 172.21.15.45:0/510750124 >> 172.21.15.45:0/510750124 conn(0x7fe064070fc0 msgr2=0x7fe0640713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.200+0000 7fe06c1e0640 1 -- 172.21.15.45:0/510750124 shutdown_connections 2023-07-29T13:27:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.200+0000 7fe06c1e0640 1 -- 172.21.15.45:0/510750124 wait complete. 2023-07-29T13:27:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.201+0000 7fe06c1e0640 10 monclient: build_initial_monmap 2023-07-29T13:27:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.201+0000 7fe06c1e0640 10 monclient: monmap: 2023-07-29T13:27:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2023-07-29T13:27:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2023-07-29T13:27:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:22.211 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.201+0000 7fe06c1e0640 1 librados: starting msgr at 2023-07-29T13:27:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.201+0000 7fe06c1e0640 1 librados: starting objecter 2023-07-29T13:27:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.201+0000 7fe06c1e0640 1 Processor -- start 2023-07-29T13:27:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.201+0000 7fe06c1e0640 1 -- start start 2023-07-29T13:27:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.201+0000 7fe06c1e0640 1 librados: setting wanted keys 2023-07-29T13:27:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.201+0000 7fe06c1e0640 1 librados: calling monclient init 2023-07-29T13:27:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.201+0000 7fe06c1e0640 10 monclient: init 2023-07-29T13:27:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T13:27:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0640a8910 0x7fe0641d8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 10 monclient(hunting): picked mon.noname-b con 0x7fe0640a8910 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0640b3ae0 0x7fe0641d9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 10 monclient(hunting): picked mon.noname-c con 0x7fe0640b3ae0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06413e3a0 0x7fe0640aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 10 monclient(hunting): picked mon.noname-a con 0x7fe06413e3a0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640ac440 con 0x7fe06413e3a0 2023-07-29T13:27:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0641d9c30 con 0x7fe0640a8910 2023-07-29T13:27:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640af690 con 0x7fe0640b3ae0 2023-07-29T13:27:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06c1e0640 10 monclient(hunting): authenticate will time out at 1365.429514s 2023-07-29T13:27:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.202+0000 7fe06a77d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06413e3a0 0x7fe0640aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06977b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0640b3ae0 0x7fe0641d9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06a77d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06413e3a0 0x7fe0640aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.45:47018/0 (socket says 172.21.15.45:47018) 2023-07-29T13:27:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06a77d640 1 -- 172.21.15.45:0/3210407586 learned_addr learned my addr 172.21.15.45:0/3210407586 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-07-29T13:27:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06a77d640 10 monclient(hunting): get_auth_request con 0x7fe06413e3a0 auth_method 0 2023-07-29T13:27:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06a77d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06a77d640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06a77d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06977b640 10 monclient(hunting): get_auth_request con 0x7fe0640b3ae0 auth_method 0 2023-07-29T13:27:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06977b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06977b640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06977b640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06a77d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06a77d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.203+0000 7fe06a77d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.204+0000 7fe06977b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.204+0000 7fe06977b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.204+0000 7fe06977b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.205+0000 7fe069f7c640 1 --2- 172.21.15.45:0/3210407586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0640a8910 0x7fe0641d8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.205+0000 7fe069f7c640 10 monclient(hunting): get_auth_request con 0x7fe0640a8910 auth_method 0 2023-07-29T13:27:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.205+0000 7fe069f7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.205+0000 7fe069f7c640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.205+0000 7fe069f7c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.205+0000 7fe069f7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.205+0000 7fe069f7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.205+0000 7fe069f7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.206+0000 7fe06a77d640 10 monclient(hunting): handle_auth_done global_id 4146 payload 824 2023-07-29T13:27:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.206+0000 7fe06a77d640 1 -- 172.21.15.45:0/3210407586 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0640b3ae0 msgr2=0x7fe0641d9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.206+0000 7fe06a77d640 1 --2- 172.21.15.45:0/3210407586 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0640b3ae0 0x7fe0641d9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.206+0000 7fe06a77d640 1 -- 172.21.15.45:0/3210407586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0640a8910 msgr2=0x7fe0641d8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.206+0000 7fe06a77d640 1 --2- 172.21.15.45:0/3210407586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0640a8910 0x7fe0641d8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.206+0000 7fe06a77d640 10 monclient: _finish_hunting 0 2023-07-29T13:27:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.206+0000 7fe06a77d640 1 monclient: found mon.noname-a 2023-07-29T13:27:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.206+0000 7fe06a77d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.206+0000 7fe06a77d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:27:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.206+0000 7fe06a77d640 1 -- 172.21.15.45:0/3210407586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c009a70 con 0x7fe06413e3a0 2023-07-29T13:27:22.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.208+0000 7fe06a77d640 1 --2- 172.21.15.45:0/3210407586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06413e3a0 0x7fe0640aad10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009eb0 tx=0x7fe05c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:22.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.208+0000 7fe05a7fc640 10 client.?.objecter ms_handle_connect 0x7fe06413e3a0 2023-07-29T13:27:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.208+0000 7fe05a7fc640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 1 -- 172.21.15.45:0/3210407586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c03e610 con 0x7fe06413e3a0 2023-07-29T13:27:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:27:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 10 monclient: dump: 2023-07-29T13:27:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:22.225 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 10 monclient: _finish_auth 0 2023-07-29T13:27:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 10 monclient: _check_auth_tickets 2023-07-29T13:27:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 1 -- 172.21.15.45:0/3210407586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c03e770 con 0x7fe06413e3a0 2023-07-29T13:27:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe06c1e0640 5 monclient: authenticate success, global_id 4146 2023-07-29T13:27:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe06c1e0640 10 monclient: _renew_subs 2023-07-29T13:27:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe06c1e0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe06c1e0640 1 -- 172.21.15.45:0/3210407586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0640af8b0 con 0x7fe06413e3a0 2023-07-29T13:27:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 1 -- 172.21.15.45:0/3210407586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c03eb20 con 0x7fe06413e3a0 2023-07-29T13:27:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:27:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05a7fc640 10 monclient: dump: 2023-07-29T13:27:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:22.228 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe05bfff640 4 set_mon_vals no callback set 2023-07-29T13:27:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe06c1e0640 10 client.4146.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe06c1e0640 10 monclient: _renew_subs 2023-07-29T13:27:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe06c1e0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe06c1e0640 1 -- 172.21.15.45:0/3210407586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0640ab480 con 0x7fe06413e3a0 2023-07-29T13:27:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.212+0000 7fe06c1e0640 1 librados: init done 2023-07-29T13:27:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.213+0000 7fe06c1e0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.213+0000 7fe06c1e0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.213+0000 7fe06c1e0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.213+0000 7fe06c1e0640 1 -- 172.21.15.45:0/3210407586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0640aaa30 con 0x7fe06413e3a0 2023-07-29T13:27:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.214+0000 7fe05a7fc640 1 -- 172.21.15.45:0/3210407586 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe05c03b050 con 0x7fe06413e3a0 2023-07-29T13:27:22.295 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:22.294+0000 7ff142a2c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-29T13:27:22.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.315+0000 7fe05a7fc640 1 -- 172.21.15.45:0/3210407586 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fe05c053480 con 0x7fe06413e3a0 2023-07-29T13:27:22.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.315+0000 7fe05a7fc640 10 client.4146.objecter ms_dispatch 0x7fe064000b90 osd_map(8..8 src has 1..8) v4 2023-07-29T13:27:22.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.315+0000 7fe05a7fc640 3 client.4146.objecter handle_osd_map got epochs [8,8] > 0 2023-07-29T13:27:22.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.315+0000 7fe05a7fc640 3 client.4146.objecter handle_osd_map decoding full epoch 8 2023-07-29T13:27:22.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.315+0000 7fe05a7fc640 20 client.4146.objecter dump_active .. 0 homeless 2023-07-29T13:27:22.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.319+0000 7fe05a7fc640 1 -- 172.21.15.45:0/3210407586 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fe0640aaa30 con 0x7fe06413e3a0 2023-07-29T13:27:22.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.319+0000 7fe05a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:22.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.319+0000 7fe05a7fc640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:22.321 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:22.319+0000 7f9c90e32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-29T13:27:22.340 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:22.339+0000 7ff142a2c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-29T13:27:22.365 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:22.363+0000 7f9c90e32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-29T13:27:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.392+0000 7fe06c1e0640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "4b9d2830-9f2c-438c-9c0b-b6692e21f921", "id": 6}] 2023-07-29T13:27:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.392+0000 7fe06c1e0640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "4b9d2830-9f2c-438c-9c0b-b6692e21f921", "id": 6}] 2023-07-29T13:27:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.392+0000 7fe06c1e0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.392+0000 7fe06c1e0640 1 -- 172.21.15.45:0/3210407586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4b9d2830-9f2c-438c-9c0b-b6692e21f921", "id": 6} v 0) v1 -- 0x7fe064067860 con 0x7fe06413e3a0 2023-07-29T13:27:22.396 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:22.393+0000 7ff142a2c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-29T13:27:22.420 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:22.419+0000 7f9c90e32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-29T13:27:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.438+0000 7fe05a7fc640 1 -- 172.21.15.45:0/3210407586 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4b9d2830-9f2c-438c-9c0b-b6692e21f921", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe05c04e060 con 0x7fe06413e3a0 2023-07-29T13:27:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.438+0000 7fe05a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "4b9d2830-9f2c-438c-9c0b-b6692e21f921", "id": 6}] 2023-07-29T13:27:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.438+0000 7fe05a7fc640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:22.440 INFO:teuthology.orchestra.run.smithi045.stdout:6 2023-07-29T13:27:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 10 librados: watch_flush enter 2023-07-29T13:27:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 10 librados: watch_flush exit 2023-07-29T13:27:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 20 client.4146.objecter shutdown clearing up homeless session... 2023-07-29T13:27:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 10 client.4146.objecter successfully canceled tick 2023-07-29T13:27:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 10 monclient: shutdown 2023-07-29T13:27:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 1 -- 172.21.15.45:0/3210407586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06413e3a0 msgr2=0x7fe0640aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 1 --2- 172.21.15.45:0/3210407586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06413e3a0 0x7fe0640aad10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009eb0 tx=0x7fe05c004a40 comp rx=0 tx=0).stop 2023-07-29T13:27:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 1 -- 172.21.15.45:0/3210407586 shutdown_connections 2023-07-29T13:27:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 1 --2- 172.21.15.45:0/3210407586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06413e3a0 0x7fe0640aad10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 1 --2- 172.21.15.45:0/3210407586 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0640b3ae0 0x7fe0641d9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 1 --2- 172.21.15.45:0/3210407586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0640a8910 0x7fe0641d8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 1 -- 172.21.15.45:0/3210407586 >> 172.21.15.45:0/3210407586 conn(0x7fe064070fc0 msgr2=0x7fe0640b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 1 -- 172.21.15.45:0/3210407586 shutdown_connections 2023-07-29T13:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 1 -- 172.21.15.45:0/3210407586 wait complete. 2023-07-29T13:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.439+0000 7fe06c1e0640 1 librados: shutdown 2023-07-29T13:27:22.443 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:22.441+0000 7ff142a2c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-29T13:27:22.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new 6593807d-34a1-431b-aa23-24e8340d887b 7 2023-07-29T13:27:22.470 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-07-29T13:27:22.469+0000 7f9c90e32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-29T13:27:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.566+0000 7f40698f5640 10 monclient: get_monmap_and_config 2023-07-29T13:27:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.566+0000 7f40698f5640 10 monclient: build_initial_monmap 2023-07-29T13:27:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.566+0000 7f40698f5640 10 monclient: monmap: 2023-07-29T13:27:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:27:22.567654+0000 2023-07-29T13:27:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:27:22.567654+0000 2023-07-29T13:27:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:22.570 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.567+0000 7f40698f5640 1 Processor -- start 2023-07-29T13:27:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.567+0000 7f40698f5640 1 -- start start 2023-07-29T13:27:22.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.567+0000 7f40698f5640 10 monclient: init 2023-07-29T13:27:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2023-07-29T13:27:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f40640b3ae0 0x7f406413dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 10 monclient(hunting): picked mon.noname-a con 0x7f40640b3ae0 addr v2:172.21.15.40:3300/0 2023-07-29T13:27:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f40640a8910 addr v1:172.21.15.45:6789/0 2023-07-29T13:27:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4064141320 0x7f406413e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 10 monclient(hunting): picked mon.noname-b con 0x7f4064141320 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40640b3eb0 con 0x7f40640a8910 2023-07-29T13:27:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40640b0f10 con 0x7f40640b3ae0 2023-07-29T13:27:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40640a9950 con 0x7f4064141320 2023-07-29T13:27:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40698f5640 10 monclient(hunting): authenticate will time out at 1365.795363s 2023-07-29T13:27:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f405a7fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f40640a8910 0x7f4064140c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52896/0 (socket says 172.21.15.45:52896) 2023-07-29T13:27:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f405a7fc640 1 -- 172.21.15.45:0/3881213696 learned_addr learned my addr 172.21.15.45:0/3881213696 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-07-29T13:27:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.568+0000 7f40637fe640 1 --2- 172.21.15.45:0/3881213696 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4064141320 0x7f406413e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.569+0000 7f40637fe640 10 monclient(hunting): get_auth_request con 0x7f4064141320 auth_method 0 2023-07-29T13:27:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.569+0000 7f40637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.569+0000 7f40637fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.569+0000 7f40637fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.569+0000 7f40637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.569+0000 7f40637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.569+0000 7f40637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:22.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.571+0000 7f40637fe640 10 monclient(hunting): handle_auth_done global_id 4133 payload 274 2023-07-29T13:27:22.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.571+0000 7f40637fe640 1 -- 172.21.15.45:0/3881213696 >> v2:172.21.15.40:3300/0 conn(0x7f40640b3ae0 msgr2=0x7f406413dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:22.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.571+0000 7f40637fe640 1 --2- 172.21.15.45:0/3881213696 >> v2:172.21.15.40:3300/0 conn(0x7f40640b3ae0 0x7f406413dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.571+0000 7f40637fe640 1 -- 172.21.15.45:0/3881213696 >> v1:172.21.15.45:6789/0 conn(0x7f40640a8910 legacy=0x7f4064140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:22.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.571+0000 7f40637fe640 10 monclient: _finish_hunting 0 2023-07-29T13:27:22.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.571+0000 7f40637fe640 1 monclient: found mon.noname-b 2023-07-29T13:27:22.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.571+0000 7f40637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:22.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.571+0000 7f40637fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:27:22.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.571+0000 7f40637fe640 1 -- 172.21.15.45:0/3881213696 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40640a9c30 con 0x7f4064141320 2023-07-29T13:27:22.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.573+0000 7f40637fe640 1 --2- 172.21.15.45:0/3881213696 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4064141320 0x7f406413e1d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f405c009d90 tx=0x7f405c02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf36b644e51963fc server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:22.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 1 -- 172.21.15.45:0/3881213696 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c02cbc0 con 0x7f4064141320 2023-07-29T13:27:22.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:22.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:27:22.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 10 monclient: dump: 2023-07-29T13:27:22.579 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:22.579 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:22.579 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.579 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.579 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:22.580 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.580 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:22.580 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:22.580 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:22.580 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 10 monclient: _finish_auth 0 2023-07-29T13:27:22.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 10 monclient: _check_auth_tickets 2023-07-29T13:27:22.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:22.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 1 -- 172.21.15.45:0/3881213696 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c02cd20 con 0x7f4064141320 2023-07-29T13:27:22.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:22.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 1 -- 172.21.15.45:0/3881213696 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c034660 con 0x7f4064141320 2023-07-29T13:27:22.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:22.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:27:22.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40627fc640 10 monclient: dump: 2023-07-29T13:27:22.581 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:22.582 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:22.582 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.582 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.582 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:22.582 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.582 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:22.582 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:22.582 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:22.583 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40698f5640 5 monclient: authenticate success, global_id 4133 2023-07-29T13:27:22.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40698f5640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:22.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.577+0000 7f40698f5640 4 set_mon_vals no callback set 2023-07-29T13:27:22.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 10 monclient: shutdown 2023-07-29T13:27:22.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:22.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 1 -- 172.21.15.45:0/3881213696 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4064141320 msgr2=0x7f406413e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:22.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 1 --2- 172.21.15.45:0/3881213696 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4064141320 0x7f406413e1d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f405c009d90 tx=0x7f405c02be10 comp rx=0 tx=0).stop 2023-07-29T13:27:22.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 1 -- 172.21.15.45:0/3881213696 shutdown_connections 2023-07-29T13:27:22.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 1 --2- 172.21.15.45:0/3881213696 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4064141320 0x7f406413e1d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 1 --2- 172.21.15.45:0/3881213696 >> v2:172.21.15.40:3300/0 conn(0x7f40640b3ae0 0x7f406413dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 1 -- 172.21.15.45:0/3881213696 >> 172.21.15.45:0/3881213696 conn(0x7f4064070fc0 msgr2=0x7f40640713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:22.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 1 -- 172.21.15.45:0/3881213696 shutdown_connections 2023-07-29T13:27:22.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 1 -- 172.21.15.45:0/3881213696 wait complete. 2023-07-29T13:27:22.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 10 monclient: build_initial_monmap 2023-07-29T13:27:22.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.578+0000 7f40698f5640 10 monclient: monmap: 2023-07-29T13:27:22.587 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2023-07-29T13:27:22.587 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:22.587 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2023-07-29T13:27:22.588 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2023-07-29T13:27:22.588 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:22.588 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.588 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:22.588 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:22.588 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:22.588 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.579+0000 7f40698f5640 1 librados: starting msgr at 2023-07-29T13:27:22.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.579+0000 7f40698f5640 1 librados: starting objecter 2023-07-29T13:27:22.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.579+0000 7f40698f5640 1 Processor -- start 2023-07-29T13:27:22.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.579+0000 7f40698f5640 1 -- start start 2023-07-29T13:27:22.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.579+0000 7f40698f5640 1 librados: setting wanted keys 2023-07-29T13:27:22.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.579+0000 7f40698f5640 1 librados: calling monclient init 2023-07-29T13:27:22.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.579+0000 7f40698f5640 10 monclient: init 2023-07-29T13:27:22.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:22.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:27:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f40640a8910 0x7f4064144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 10 monclient(hunting): picked mon.noname-c con 0x7f40640a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40640b3ae0 0x7f4064138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 10 monclient(hunting): picked mon.noname-b con 0x7f40640b3ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4064141320 0x7f40641d9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 10 monclient(hunting): picked mon.noname-a con 0x7f4064141320 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4064144dd0 con 0x7f4064141320 2023-07-29T13:27:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641398d0 con 0x7f40640b3ae0 2023-07-29T13:27:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4064139440 con 0x7f40640a8910 2023-07-29T13:27:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f40698f5640 10 monclient(hunting): authenticate will time out at 1365.807196s 2023-07-29T13:27:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f405a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40640b3ae0 0x7f4064138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f4062ffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f40640a8910 0x7f4064144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f405a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40640b3ae0 0x7f4064138ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47800/0 (socket says 172.21.15.45:47800) 2023-07-29T13:27:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f405a7fc640 1 -- 172.21.15.45:0/2114452111 learned_addr learned my addr 172.21.15.45:0/2114452111 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-07-29T13:27:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f405a7fc640 10 monclient(hunting): get_auth_request con 0x7f40640b3ae0 auth_method 0 2023-07-29T13:27:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f405a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f405a7fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.580+0000 7f405a7fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.581+0000 7f4062ffd640 10 monclient(hunting): get_auth_request con 0x7f40640a8910 auth_method 0 2023-07-29T13:27:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.581+0000 7f4062ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.581+0000 7f4062ffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.581+0000 7f4062ffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.581+0000 7f405a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.581+0000 7f405a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.581+0000 7f405a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.581+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.581+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.581+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.584+0000 7f405a7fc640 10 monclient(hunting): handle_auth_done global_id 4150 payload 824 2023-07-29T13:27:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.584+0000 7f405a7fc640 1 -- 172.21.15.45:0/2114452111 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f40640a8910 msgr2=0x7f4064144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.584+0000 7f405a7fc640 1 --2- 172.21.15.45:0/2114452111 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f40640a8910 0x7f4064144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.584+0000 7f405a7fc640 1 -- 172.21.15.45:0/2114452111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4064141320 msgr2=0x7f40641d9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.584+0000 7f405a7fc640 1 --2- 172.21.15.45:0/2114452111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4064141320 0x7f40641d9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.584+0000 7f405a7fc640 10 monclient: _finish_hunting 0 2023-07-29T13:27:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.584+0000 7f405a7fc640 1 monclient: found mon.noname-b 2023-07-29T13:27:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.584+0000 7f405a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.584+0000 7f405a7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:27:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.584+0000 7f405a7fc640 1 -- 172.21.15.45:0/2114452111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c009a70 con 0x7f40640b3ae0 2023-07-29T13:27:22.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.586+0000 7f405a7fc640 1 --2- 172.21.15.45:0/2114452111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40640b3ae0 0x7f4064138ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f404c00c4a0 tx=0x7f404c00c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.586+0000 7f4060ff9640 10 client.?.objecter ms_handle_connect 0x7f40640b3ae0 2023-07-29T13:27:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.586+0000 7f4060ff9640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.589+0000 7f4060ff9640 1 -- 172.21.15.45:0/2114452111 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c005ce0 con 0x7f40640b3ae0 2023-07-29T13:27:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.589+0000 7f4060ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.589+0000 7f4060ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:27:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.589+0000 7f4060ff9640 10 monclient: dump: 2023-07-29T13:27:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:22.598 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f4060ff9640 10 monclient: _finish_auth 0 2023-07-29T13:27:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f4060ff9640 10 monclient: _check_auth_tickets 2023-07-29T13:27:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f4060ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f4060ff9640 1 -- 172.21.15.45:0/2114452111 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404c005e40 con 0x7f40640b3ae0 2023-07-29T13:27:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 5 monclient: authenticate success, global_id 4150 2023-07-29T13:27:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 10 monclient: _renew_subs 2023-07-29T13:27:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:22.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 1 -- 172.21.15.45:0/2114452111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4064138820 con 0x7f40640b3ae0 2023-07-29T13:27:22.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f4060ff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:22.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f4060ff9640 1 -- 172.21.15.45:0/2114452111 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c0072d0 con 0x7f40640b3ae0 2023-07-29T13:27:22.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f4060ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:22.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f4060ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:22.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f4060ff9640 10 monclient: dump: 2023-07-29T13:27:22.601 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2023-07-29T13:27:22.601 INFO:teuthology.orchestra.run.smithi045.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:22.601 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.601 INFO:teuthology.orchestra.run.smithi045.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:22.601 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:22.601 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2023-07-29T13:27:22.601 INFO:teuthology.orchestra.run.smithi045.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:22.601 INFO:teuthology.orchestra.run.smithi045.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:22.602 INFO:teuthology.orchestra.run.smithi045.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:22.602 INFO:teuthology.orchestra.run.smithi045.stderr: 2023-07-29T13:27:22.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f4061ffb640 4 set_mon_vals no callback set 2023-07-29T13:27:22.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 10 client.4150.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:22.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 10 monclient: _renew_subs 2023-07-29T13:27:22.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:22.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 1 -- 172.21.15.45:0/2114452111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40641d9840 con 0x7f40640b3ae0 2023-07-29T13:27:22.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 1 librados: init done 2023-07-29T13:27:22.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:22.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:22.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:22.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.590+0000 7f40698f5640 1 -- 172.21.15.45:0/2114452111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40641395f0 con 0x7f40640b3ae0 2023-07-29T13:27:22.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.593+0000 7f4060ff9640 1 -- 172.21.15.45:0/2114452111 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f404c01c440 con 0x7f40640b3ae0 2023-07-29T13:27:22.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.593+0000 7f4060ff9640 1 -- 172.21.15.45:0/2114452111 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f404c01ba10 con 0x7f40640b3ae0 2023-07-29T13:27:22.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.593+0000 7f4060ff9640 10 client.4150.objecter ms_dispatch 0x7f4064000b90 osd_map(9..9 src has 1..9) v4 2023-07-29T13:27:22.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.593+0000 7f4060ff9640 3 client.4150.objecter handle_osd_map got epochs [9,9] > 0 2023-07-29T13:27:22.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.593+0000 7f4060ff9640 3 client.4150.objecter handle_osd_map decoding full epoch 9 2023-07-29T13:27:22.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.593+0000 7f4060ff9640 20 client.4150.objecter dump_active .. 0 homeless 2023-07-29T13:27:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.694+0000 7f4060ff9640 1 -- 172.21.15.45:0/2114452111 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f40641395f0 con 0x7f40640b3ae0 2023-07-29T13:27:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.694+0000 7f4060ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.694+0000 7f4060ff9640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:22.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.767+0000 7f40698f5640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "6593807d-34a1-431b-aa23-24e8340d887b", "id": 7}] 2023-07-29T13:27:22.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.767+0000 7f40698f5640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "6593807d-34a1-431b-aa23-24e8340d887b", "id": 7}] 2023-07-29T13:27:22.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.767+0000 7f40698f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:22.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.767+0000 7f40698f5640 1 -- 172.21.15.45:0/2114452111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6593807d-34a1-431b-aa23-24e8340d887b", "id": 7} v 0) v1 -- 0x7f40640b58d0 con 0x7f40640b3ae0 2023-07-29T13:27:22.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:22.811+0000 7f4060ff9640 1 -- 172.21.15.45:0/2114452111 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 2) v1 ==== 44056+0+0 (secure 0 0 0) 0x7f404c01bca0 con 0x7f40640b3ae0 2023-07-29T13:27:22.920 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:22.919+0000 7ff13cb2a640 -1 mgr handle_mgr_map I was active but no longer am 2023-07-29T13:27:23.067 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:23.066+0000 7f403e455040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-29T13:27:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.218+0000 7f4060ff9640 1 -- 172.21.15.45:0/2114452111 <== mon.1 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6593807d-34a1-431b-aa23-24e8340d887b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f404c017020 con 0x7f40640b3ae0 2023-07-29T13:27:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.218+0000 7f4060ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "6593807d-34a1-431b-aa23-24e8340d887b", "id": 7}] 2023-07-29T13:27:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.218+0000 7f4060ff9640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:23.221 INFO:teuthology.orchestra.run.smithi045.stdout:7 2023-07-29T13:27:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 10 librados: watch_flush enter 2023-07-29T13:27:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 10 librados: watch_flush exit 2023-07-29T13:27:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 20 client.4150.objecter shutdown clearing up homeless session... 2023-07-29T13:27:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 10 client.4150.objecter successfully canceled tick 2023-07-29T13:27:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 10 monclient: shutdown 2023-07-29T13:27:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 1 -- 172.21.15.45:0/2114452111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40640b3ae0 msgr2=0x7f4064138ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 1 --2- 172.21.15.45:0/2114452111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40640b3ae0 0x7f4064138ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f404c00c4a0 tx=0x7f404c00c8a0 comp rx=0 tx=0).stop 2023-07-29T13:27:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 1 -- 172.21.15.45:0/2114452111 shutdown_connections 2023-07-29T13:27:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 1 --2- 172.21.15.45:0/2114452111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40640b3ae0 0x7f4064138ce0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 1 --2- 172.21.15.45:0/2114452111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4064141320 0x7f40641d9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 1 --2- 172.21.15.45:0/2114452111 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f40640a8910 0x7f4064144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.219+0000 7f40698f5640 1 -- 172.21.15.45:0/2114452111 >> 172.21.15.45:0/2114452111 conn(0x7f4064070fc0 msgr2=0x7f406413df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.220+0000 7f40698f5640 1 -- 172.21.15.45:0/2114452111 shutdown_connections 2023-07-29T13:27:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.220+0000 7f40698f5640 1 -- 172.21.15.45:0/2114452111 wait complete. 2023-07-29T13:27:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-07-29T13:27:23.220+0000 7f40698f5640 1 librados: shutdown 2023-07-29T13:27:23.235 DEBUG:tasks.ceph_manager:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-07-29T13:27:23.235 INFO:tasks.ceph.osd.0:Restarting daemon 2023-07-29T13:27:23.235 DEBUG:teuthology.orchestra.run.smithi040:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 2023-07-29T13:27:23.238 INFO:tasks.ceph.osd.0:Started 2023-07-29T13:27:23.238 DEBUG:tasks.ceph_manager:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-07-29T13:27:23.238 INFO:tasks.ceph.osd.1:Restarting daemon 2023-07-29T13:27:23.239 DEBUG:teuthology.orchestra.run.smithi040:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 2023-07-29T13:27:23.240 INFO:tasks.ceph.osd.1:Started 2023-07-29T13:27:23.241 DEBUG:tasks.ceph_manager:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-07-29T13:27:23.241 INFO:tasks.ceph.osd.2:Restarting daemon 2023-07-29T13:27:23.241 DEBUG:teuthology.orchestra.run.smithi040:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 2023-07-29T13:27:23.243 INFO:tasks.ceph.osd.2:Started 2023-07-29T13:27:23.243 DEBUG:tasks.ceph_manager:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-07-29T13:27:23.244 INFO:tasks.ceph.osd.3:Restarting daemon 2023-07-29T13:27:23.244 DEBUG:teuthology.orchestra.run.smithi040:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 2023-07-29T13:27:23.246 INFO:tasks.ceph.osd.3:Started 2023-07-29T13:27:23.246 DEBUG:tasks.ceph_manager:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-07-29T13:27:23.246 INFO:tasks.ceph.osd.4:Restarting daemon 2023-07-29T13:27:23.247 DEBUG:teuthology.orchestra.run.smithi045:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 2023-07-29T13:27:23.249 INFO:tasks.ceph.osd.4:Started 2023-07-29T13:27:23.249 DEBUG:tasks.ceph_manager:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-07-29T13:27:23.249 INFO:tasks.ceph.osd.5:Restarting daemon 2023-07-29T13:27:23.249 DEBUG:teuthology.orchestra.run.smithi045:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 2023-07-29T13:27:23.251 INFO:tasks.ceph.osd.5:Started 2023-07-29T13:27:23.252 DEBUG:tasks.ceph_manager:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-07-29T13:27:23.252 INFO:tasks.ceph.osd.6:Restarting daemon 2023-07-29T13:27:23.252 DEBUG:teuthology.orchestra.run.smithi045:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 2023-07-29T13:27:23.254 INFO:tasks.ceph.osd.6:Started 2023-07-29T13:27:23.255 DEBUG:tasks.ceph_manager:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-07-29T13:27:23.255 INFO:tasks.ceph.osd.7:Restarting daemon 2023-07-29T13:27:23.255 DEBUG:teuthology.orchestra.run.smithi045:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 2023-07-29T13:27:23.257 INFO:tasks.ceph.osd.7:Started 2023-07-29T13:27:23.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-29T13:27:23.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.381+0000 7f6319697640 10 monclient: get_monmap_and_config 2023-07-29T13:27:23.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.381+0000 7f6319697640 10 monclient: build_initial_monmap 2023-07-29T13:27:23.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.382+0000 7f6319697640 10 monclient: monmap: 2023-07-29T13:27:23.387 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:27:23.387 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:27:23.383195+0000 2023-07-29T13:27:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:27:23.383195+0000 2023-07-29T13:27:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:23.390 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.382+0000 7f6319697640 1 Processor -- start 2023-07-29T13:27:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.382+0000 7f6319697640 1 -- start start 2023-07-29T13:27:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.382+0000 7f6319697640 10 monclient: init 2023-07-29T13:27:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2023-07-29T13:27:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f63140b3ae0 addr v1:172.21.15.45:6789/0 2023-07-29T13:27:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f631413e3a0 0x7f6314140c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 10 monclient(hunting): picked mon.noname-a con 0x7f631413e3a0 addr v2:172.21.15.40:3300/0 2023-07-29T13:27:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f63140a8910 0x7f6314141150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 10 monclient(hunting): picked mon.noname-b con 0x7f63140a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63140b3eb0 con 0x7f63140b3ae0 2023-07-29T13:27:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63140b0f10 con 0x7f631413e3a0 2023-07-29T13:27:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63140a9950 con 0x7f63140a8910 2023-07-29T13:27:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.383+0000 7f6319697640 10 monclient(hunting): authenticate will time out at 1365.517033s 2023-07-29T13:27:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f63137fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f63140a8910 0x7f6314141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f63127fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f631413e3a0 0x7f6314140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f6312ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f63140b3ae0 0x7f631413dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.40:54400/0 (socket says 172.21.15.40:54400) 2023-07-29T13:27:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f6312ffd640 1 -- 172.21.15.40:0/1486020895 learned_addr learned my addr 172.21.15.40:0/1486020895 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:27:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f63127fc640 10 monclient(hunting): get_auth_request con 0x7f631413e3a0 auth_method 0 2023-07-29T13:27:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f63127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f63127fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f63127fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f63137fe640 10 monclient(hunting): get_auth_request con 0x7f63140a8910 auth_method 0 2023-07-29T13:27:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f63137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f63137fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.384+0000 7f63137fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:23.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.385+0000 7f63127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:23.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.385+0000 7f63127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.385+0000 7f63127fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.385+0000 7f63137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.385+0000 7f63137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.385+0000 7f63137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.387+0000 7f63127fc640 10 monclient(hunting): handle_auth_done global_id 4155 payload 274 2023-07-29T13:27:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.387+0000 7f63127fc640 1 -- 172.21.15.40:0/1486020895 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f63140a8910 msgr2=0x7f6314141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.387+0000 7f63127fc640 1 --2- 172.21.15.40:0/1486020895 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f63140a8910 0x7f6314141150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.387+0000 7f63127fc640 1 -- 172.21.15.40:0/1486020895 >> v1:172.21.15.45:6789/0 conn(0x7f63140b3ae0 legacy=0x7f631413dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.387+0000 7f63127fc640 10 monclient: _finish_hunting 0 2023-07-29T13:27:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.387+0000 7f63127fc640 1 monclient: found mon.noname-a 2023-07-29T13:27:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.387+0000 7f63127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.387+0000 7f63127fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:27:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.387+0000 7f63127fc640 1 -- 172.21.15.40:0/1486020895 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6314141690 con 0x7f631413e3a0 2023-07-29T13:27:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.389+0000 7f63127fc640 1 --2- 172.21.15.40:0/1486020895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631413e3a0 0x7f6314140c10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f63080097a0 tx=0x7f630802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf924efd3ef51f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 1 -- 172.21.15.40:0/1486020895 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63080317c0 con 0x7f631413e3a0 2023-07-29T13:27:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:27:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 10 monclient: dump: 2023-07-29T13:27:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:27:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:23.399 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:23.399 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:23.399 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:23.399 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:23.399 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:23.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 10 monclient: _finish_auth 0 2023-07-29T13:27:23.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 10 monclient: _check_auth_tickets 2023-07-29T13:27:23.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:23.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 1 -- 172.21.15.40:0/1486020895 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6308031920 con 0x7f631413e3a0 2023-07-29T13:27:23.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:23.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 1 -- 172.21.15.40:0/1486020895 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308031cf0 con 0x7f631413e3a0 2023-07-29T13:27:23.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:23.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:27:23.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6311ffb640 10 monclient: dump: 2023-07-29T13:27:23.400 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:27:23.401 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:23.401 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:23.401 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:23.401 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:23.401 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:23.401 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:23.401 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:23.401 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:23.402 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6319697640 5 monclient: authenticate success, global_id 4155 2023-07-29T13:27:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6319697640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.393+0000 7f6319697640 4 set_mon_vals no callback set 2023-07-29T13:27:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.394+0000 7f6319697640 10 monclient: shutdown 2023-07-29T13:27:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.394+0000 7f6319697640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.394+0000 7f6319697640 1 -- 172.21.15.40:0/1486020895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631413e3a0 msgr2=0x7f6314140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.394+0000 7f6319697640 1 --2- 172.21.15.40:0/1486020895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631413e3a0 0x7f6314140c10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f63080097a0 tx=0x7f630802b860 comp rx=0 tx=0).stop 2023-07-29T13:27:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.394+0000 7f6319697640 1 -- 172.21.15.40:0/1486020895 shutdown_connections 2023-07-29T13:27:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.394+0000 7f6319697640 1 --2- 172.21.15.40:0/1486020895 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f63140a8910 0x7f6314141150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.394+0000 7f6319697640 1 --2- 172.21.15.40:0/1486020895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631413e3a0 0x7f6314140c10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.394+0000 7f6319697640 1 -- 172.21.15.40:0/1486020895 >> 172.21.15.40:0/1486020895 conn(0x7f6314070fc0 msgr2=0x7f63140713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.394+0000 7f6319697640 1 -- 172.21.15.40:0/1486020895 shutdown_connections 2023-07-29T13:27:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.394+0000 7f6319697640 1 -- 172.21.15.40:0/1486020895 wait complete. 2023-07-29T13:27:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.395+0000 7f6319697640 10 monclient: build_initial_monmap 2023-07-29T13:27:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.395+0000 7f6319697640 10 monclient: monmap: 2023-07-29T13:27:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:27:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:27:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:27:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:23.405 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.395+0000 7f6319697640 1 librados: starting msgr at 2023-07-29T13:27:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.395+0000 7f6319697640 1 librados: starting objecter 2023-07-29T13:27:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 1 Processor -- start 2023-07-29T13:27:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 1 -- start start 2023-07-29T13:27:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 1 librados: setting wanted keys 2023-07-29T13:27:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 1 librados: calling monclient init 2023-07-29T13:27:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 10 monclient: init 2023-07-29T13:27:23.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:23.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:27:23.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f63140a8910 0x7f63140ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:23.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 10 monclient(hunting): picked mon.noname-c con 0x7f63140a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:23.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140b3ae0 0x7f63140ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 10 monclient(hunting): picked mon.noname-b con 0x7f63140b3ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631413e3a0 0x7f63140af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 10 monclient(hunting): picked mon.noname-a con 0x7f631413e3a0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631413ad60 con 0x7f631413e3a0 2023-07-29T13:27:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63140adc10 con 0x7f63140b3ae0 2023-07-29T13:27:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63140af2d0 con 0x7f63140a8910 2023-07-29T13:27:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.396+0000 7f6319697640 10 monclient(hunting): authenticate will time out at 1365.530068s 2023-07-29T13:27:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.397+0000 7f63137fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631413e3a0 0x7f63140af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.397+0000 7f63127fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140b3ae0 0x7f63140ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.397+0000 7f6312ffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f63140a8910 0x7f63140ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f6312ffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f63140a8910 0x7f63140ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:45514/0 (socket says 172.21.15.40:45514) 2023-07-29T13:27:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f63137fe640 10 monclient(hunting): get_auth_request con 0x7f631413e3a0 auth_method 0 2023-07-29T13:27:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f6312ffd640 1 -- 172.21.15.40:0/4054901548 learned_addr learned my addr 172.21.15.40:0/4054901548 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:27:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f63137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f63137fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f63137fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f6312ffd640 10 monclient(hunting): get_auth_request con 0x7f63140a8910 auth_method 0 2023-07-29T13:27:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f6312ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f6312ffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f6312ffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f63127fc640 10 monclient(hunting): get_auth_request con 0x7f63140b3ae0 auth_method 0 2023-07-29T13:27:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f63127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f63127fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f63127fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f63137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f63137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.398+0000 7f63137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:23.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.399+0000 7f63127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:23.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.399+0000 7f63127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:23.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.399+0000 7f63127fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:23.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.399+0000 7f6312ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:23.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.399+0000 7f6312ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:23.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.399+0000 7f6312ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:23.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.403+0000 7f63127fc640 10 monclient(hunting): handle_auth_done global_id 4162 payload 824 2023-07-29T13:27:23.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.403+0000 7f63127fc640 1 -- 172.21.15.40:0/4054901548 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f63140a8910 msgr2=0x7f63140ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:23.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.403+0000 7f63127fc640 1 --2- 172.21.15.40:0/4054901548 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f63140a8910 0x7f63140ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:23.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.403+0000 7f63127fc640 1 -- 172.21.15.40:0/4054901548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631413e3a0 msgr2=0x7f63140af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:23.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.403+0000 7f63127fc640 1 --2- 172.21.15.40:0/4054901548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631413e3a0 0x7f63140af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:23.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.403+0000 7f63127fc640 10 monclient: _finish_hunting 0 2023-07-29T13:27:23.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.403+0000 7f63127fc640 1 monclient: found mon.noname-b 2023-07-29T13:27:23.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.403+0000 7f63127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:23.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.403+0000 7f63127fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:27:23.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.403+0000 7f63127fc640 1 -- 172.21.15.40:0/4054901548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6308009400 con 0x7f63140b3ae0 2023-07-29T13:27:23.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.406+0000 7f63127fc640 1 --2- 172.21.15.40:0/4054901548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140b3ae0 0x7f63140ac920 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6308033000 tx=0x7f63080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:23.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.406+0000 7f62fb7fe640 10 client.?.objecter ms_handle_connect 0x7f63140b3ae0 2023-07-29T13:27:23.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.406+0000 7f62fb7fe640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:23.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.409+0000 7f62fb7fe640 1 -- 172.21.15.40:0/4054901548 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308032860 con 0x7f63140b3ae0 2023-07-29T13:27:23.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.409+0000 7f62fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:23.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.409+0000 7f62fb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:27:23.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.409+0000 7f62fb7fe640 10 monclient: dump: 2023-07-29T13:27:23.417 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:27:23.417 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:23.417 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:23.417 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:23.418 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.409+0000 7f62fb7fe640 10 monclient: _finish_auth 0 2023-07-29T13:27:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.410+0000 7f62fb7fe640 10 monclient: _check_auth_tickets 2023-07-29T13:27:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.410+0000 7f62fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.410+0000 7f6319697640 5 monclient: authenticate success, global_id 4162 2023-07-29T13:27:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.410+0000 7f6319697640 10 monclient: _renew_subs 2023-07-29T13:27:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.410+0000 7f6319697640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.410+0000 7f6319697640 1 -- 172.21.15.40:0/4054901548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63140ace60 con 0x7f63140b3ae0 2023-07-29T13:27:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.410+0000 7f6319697640 10 client.4162.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.410+0000 7f6319697640 10 monclient: _renew_subs 2023-07-29T13:27:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.410+0000 7f6319697640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.410+0000 7f6319697640 1 -- 172.21.15.40:0/4054901548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63140afd30 con 0x7f63140b3ae0 2023-07-29T13:27:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.410+0000 7f6319697640 1 librados: init done 2023-07-29T13:27:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.411+0000 7f62f97fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.411+0000 7f62f97fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.411+0000 7f62f97fa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.411+0000 7f62f97fa640 1 -- 172.21.15.40:0/4054901548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f4005040 con 0x7f63140b3ae0 2023-07-29T13:27:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.411+0000 7f62fb7fe640 1 -- 172.21.15.40:0/4054901548 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63080329c0 con 0x7f63140b3ae0 2023-07-29T13:27:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.411+0000 7f62fb7fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.411+0000 7f62fb7fe640 1 -- 172.21.15.40:0/4054901548 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308032d50 con 0x7f63140b3ae0 2023-07-29T13:27:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.411+0000 7f62fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.411+0000 7f62fb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.411+0000 7f62fb7fe640 10 monclient: dump: 2023-07-29T13:27:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:27:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:23.423 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:27:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.411+0000 7f63117fa640 4 set_mon_vals no callback set 2023-07-29T13:27:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.596+0000 7f62fb7fe640 1 -- 172.21.15.40:0/4054901548 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 2) v1 ==== 44056+0+0 (secure 0 0 0) 0x7f630803a380 con 0x7f63140b3ae0 2023-07-29T13:27:23.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.605+0000 7f62fb7fe640 1 -- 172.21.15.40:0/4054901548 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f63080394d0 con 0x7f63140b3ae0 2023-07-29T13:27:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.605+0000 7f62fb7fe640 10 client.4162.objecter ms_dispatch 0x7f6314000b90 osd_map(10..10 src has 1..10) v4 2023-07-29T13:27:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.605+0000 7f62fb7fe640 3 client.4162.objecter handle_osd_map got epochs [10,10] > 0 2023-07-29T13:27:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.605+0000 7f62fb7fe640 3 client.4162.objecter handle_osd_map decoding full epoch 10 2023-07-29T13:27:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.605+0000 7f62fb7fe640 20 client.4162.objecter dump_active .. 0 homeless 2023-07-29T13:27:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.605+0000 7f62fb7fe640 1 -- 172.21.15.40:0/4054901548 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f630803a630 con 0x7f63140b3ae0 2023-07-29T13:27:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.605+0000 7f62fb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.605+0000 7f62fb7fe640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:23.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.728+0000 7f62f97fa640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:23.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.728+0000 7f62f97fa640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:23.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.728+0000 7f62f97fa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:23.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.728+0000 7f62f97fa640 1 -- 172.21.15.40:0/4054901548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62f40055c0 con 0x7f63140b3ae0 2023-07-29T13:27:23.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:23.946+0000 7f62fb7fe640 1 -- 172.21.15.40:0/4054901548 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 3) v1 ==== 87920+0+0 (secure 0 0 0) 0x7f63080443a0 con 0x7f63140b3ae0 2023-07-29T13:27:24.000 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:23.999+0000 7f403e455040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-29T13:27:24.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.022+0000 7f62fb7fe640 1 -- 172.21.15.40:0/4054901548 <== mon.1 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f6308063e60 con 0x7f63140b3ae0 2023-07-29T13:27:24.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.022+0000 7f62fb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:24.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.022+0000 7f62fb7fe640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:24.025 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:27:24.025 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","created":"2023-07-29T13:27:17.185104+0000","modified":"2023-07-29T13:27:22.831496+0000","last_up_change":"0.000000","last_in_change":"2023-07-29T13:27:22.831496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6069a944-26b4-4b00-94e2-f91c485e8957","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab18edd4-e1cb-4815-b5a8-af2b4b2e790b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4575bc2c-4d43-46d3-95ba-1dc2c94278e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e7224640-7d33-4bb7-81a5-99fc26da7763","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23cd76f6-788b-48cf-9438-e931659d3a1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0ddb1e3f-0f65-40d6-ab50-5dc521774445","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4b9d2830-9f2c-438c-9c0b-b6692e21f921","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6593807d-34a1-431b-aa23-24e8340d887b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-29T13:27:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 10 librados: watch_flush enter 2023-07-29T13:27:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 10 librados: watch_flush exit 2023-07-29T13:27:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 20 client.4162.objecter shutdown clearing up homeless session... 2023-07-29T13:27:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 10 client.4162.objecter successfully canceled tick 2023-07-29T13:27:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 10 monclient: shutdown 2023-07-29T13:27:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 1 -- 172.21.15.40:0/4054901548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140b3ae0 msgr2=0x7f63140ac920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 1 --2- 172.21.15.40:0/4054901548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140b3ae0 0x7f63140ac920 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6308033000 tx=0x7f63080046c0 comp rx=0 tx=0).stop 2023-07-29T13:27:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 1 -- 172.21.15.40:0/4054901548 shutdown_connections 2023-07-29T13:27:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 1 --2- 172.21.15.40:0/4054901548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140b3ae0 0x7f63140ac920 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 1 --2- 172.21.15.40:0/4054901548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631413e3a0 0x7f63140af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 1 --2- 172.21.15.40:0/4054901548 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f63140a8910 0x7f63140ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 1 -- 172.21.15.40:0/4054901548 >> 172.21.15.40:0/4054901548 conn(0x7f6314070fc0 msgr2=0x7f631413d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 1 -- 172.21.15.40:0/4054901548 shutdown_connections 2023-07-29T13:27:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.023+0000 7f62f97fa640 1 -- 172.21.15.40:0/4054901548 wait complete. 2023-07-29T13:27:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:27:24.024+0000 7f62f97fa640 1 librados: shutdown 2023-07-29T13:27:24.041 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-07-29T13:27:24.042 INFO:tasks.ceph:Waiting for OSDs to come up 2023-07-29T13:27:24.189 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:24.188+0000 7f403e455040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-29T13:27:24.273 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:24.272+0000 7f403e455040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-29T13:27:24.347 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:24.346+0000 7f403e455040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-29T13:27:24.847 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-29T13:27:24.993 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.991+0000 7f6c7038e640 10 monclient: get_monmap_and_config 2023-07-29T13:27:24.993 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.991+0000 7f6c7038e640 10 monclient: build_initial_monmap 2023-07-29T13:27:24.993 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.991+0000 7f6c7038e640 10 monclient: monmap: 2023-07-29T13:27:24.993 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:27:24.993 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:24.993 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:27:24.992422+0000 2023-07-29T13:27:24.994 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:27:24.992422+0000 2023-07-29T13:27:24.994 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:24.994 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:24.994 INFO:teuthology.misc.health.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:24.994 INFO:teuthology.misc.health.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:24.994 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:24.994 INFO:teuthology.misc.health.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:24.995 INFO:teuthology.misc.health.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:24.995 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:24.996 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.992+0000 7f6c7038e640 1 Processor -- start 2023-07-29T13:27:24.996 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.992+0000 7f6c7038e640 1 -- start start 2023-07-29T13:27:24.996 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.992+0000 7f6c7038e640 10 monclient: init 2023-07-29T13:27:24.996 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:24.996 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 10 monclient: _add_conns ranks=[3,4,0,2,1] 2023-07-29T13:27:24.996 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c680b1a70 0x7f6c680b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:24.997 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 10 monclient(hunting): picked mon.noname-a con 0x7f6c680b1a70 addr v2:172.21.15.40:3300/0 2023-07-29T13:27:24.997 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6c6813dcb0 0x7f6c6813e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:24.997 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 10 monclient(hunting): picked mon.noname-c con 0x7f6c6813dcb0 addr v2:172.21.15.45:3300/0 2023-07-29T13:27:24.997 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6c68140da0 addr v1:172.21.15.40:6789/0 2023-07-29T13:27:24.997 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6813e650 con 0x7f6c68140da0 2023-07-29T13:27:24.997 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:24.997 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c680a9540 con 0x7f6c680b1a70 2023-07-29T13:27:24.997 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:24.998 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c680a9740 con 0x7f6c6813dcb0 2023-07-29T13:27:24.998 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:24.998 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.993+0000 7f6c7038e640 10 monclient(hunting): authenticate will time out at 1367.126505s 2023-07-29T13:27:24.998 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.994+0000 7f6c6e12a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c680b1a70 0x7f6c680b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:24.999 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.994+0000 7f6c6e92b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6c68140da0 0x7f6c681411e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40832/0 (socket says 172.21.15.40:40832) 2023-07-29T13:27:24.999 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.994+0000 7f6c6d929640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6c6813dcb0 0x7f6c6813e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:24.999 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.994+0000 7f6c6e92b640 1 -- 172.21.15.40:0/1338166505 learned_addr learned my addr 172.21.15.40:0/1338166505 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:27:24.999 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.995+0000 7f6c6d929640 10 monclient(hunting): get_auth_request con 0x7f6c6813dcb0 auth_method 0 2023-07-29T13:27:24.999 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.995+0000 7f6c6d929640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:24.999 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.995+0000 7f6c6d929640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:24.999 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.995+0000 7f6c6d929640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:25.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.995+0000 7f6c6e12a640 10 monclient(hunting): get_auth_request con 0x7f6c680b1a70 auth_method 0 2023-07-29T13:27:25.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.995+0000 7f6c6e12a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:25.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.995+0000 7f6c6e12a640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:25.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.995+0000 7f6c6e12a640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:25.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.995+0000 7f6c6d929640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:25.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.995+0000 7f6c6d929640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:25.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.995+0000 7f6c6d929640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:25.001 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.996+0000 7f6c6e12a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:25.001 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.996+0000 7f6c6e12a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:25.001 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.996+0000 7f6c6e12a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:25.001 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.998+0000 7f6c6d929640 10 monclient(hunting): handle_auth_done global_id 4165 payload 274 2023-07-29T13:27:25.001 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.998+0000 7f6c6d929640 1 -- 172.21.15.40:0/1338166505 >> v2:172.21.15.40:3300/0 conn(0x7f6c680b1a70 msgr2=0x7f6c680b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:25.002 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.998+0000 7f6c6d929640 1 --2- 172.21.15.40:0/1338166505 >> v2:172.21.15.40:3300/0 conn(0x7f6c680b1a70 0x7f6c680b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:25.002 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.998+0000 7f6c6d929640 1 -- 172.21.15.40:0/1338166505 >> v1:172.21.15.40:6789/0 conn(0x7f6c68140da0 legacy=0x7f6c681411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:25.002 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.998+0000 7f6c6d929640 10 monclient: _finish_hunting 0 2023-07-29T13:27:25.002 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.998+0000 7f6c6d929640 1 monclient: found mon.noname-c 2023-07-29T13:27:25.002 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.998+0000 7f6c6d929640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:25.002 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.998+0000 7f6c6d929640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:27:25.002 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:24.998+0000 7f6c6d929640 1 -- 172.21.15.40:0/1338166505 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c680a9a20 con 0x7f6c6813dcb0 2023-07-29T13:27:25.003 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.001+0000 7f6c6d929640 1 --2- 172.21.15.40:0/1338166505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6813dcb0 0x7f6c6813e110 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6c68066610 tx=0x7f6c5802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88bf1d86535c728b server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:25.009 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 1 -- 172.21.15.40:0/1338166505 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5802e070 con 0x7f6c6813dcb0 2023-07-29T13:27:25.009 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:25.009 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:27:25.009 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 10 monclient: dump: 2023-07-29T13:27:25.009 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:25.010 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:25.010 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:25.010 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:25.010 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:25.010 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:25.010 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:25.010 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:25.010 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:25.011 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:25.011 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 10 monclient: _finish_auth 0 2023-07-29T13:27:25.011 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 10 monclient: _check_auth_tickets 2023-07-29T13:27:25.011 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:25.011 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 1 -- 172.21.15.40:0/1338166505 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c580315a0 con 0x7f6c6813dcb0 2023-07-29T13:27:25.011 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c7038e640 5 monclient: authenticate success, global_id 4165 2023-07-29T13:27:25.011 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c7038e640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:27:25.011 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:25.012 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 1 -- 172.21.15.40:0/1338166505 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58031930 con 0x7f6c6813dcb0 2023-07-29T13:27:25.012 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c7038e640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:25.012 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c7038e640 4 set_mon_vals no callback set 2023-07-29T13:27:25.012 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:25.012 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:25.012 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.007+0000 7f6c6d128640 10 monclient: dump: 2023-07-29T13:27:25.012 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:25.013 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:25.013 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:25.013 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:25.013 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:25.013 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:25.013 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:25.013 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:25.013 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:25.014 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:25.016 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.008+0000 7f6c7038e640 10 monclient: shutdown 2023-07-29T13:27:25.016 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.008+0000 7f6c7038e640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:25.016 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.008+0000 7f6c7038e640 1 -- 172.21.15.40:0/1338166505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6813dcb0 msgr2=0x7f6c6813e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:25.016 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.008+0000 7f6c7038e640 1 --2- 172.21.15.40:0/1338166505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6813dcb0 0x7f6c6813e110 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6c68066610 tx=0x7f6c5802b840 comp rx=0 tx=0).stop 2023-07-29T13:27:25.016 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.008+0000 7f6c7038e640 1 -- 172.21.15.40:0/1338166505 shutdown_connections 2023-07-29T13:27:25.017 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.008+0000 7f6c7038e640 1 --2- 172.21.15.40:0/1338166505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6813dcb0 0x7f6c6813e110 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:25.017 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.008+0000 7f6c7038e640 1 --2- 172.21.15.40:0/1338166505 >> v2:172.21.15.40:3300/0 conn(0x7f6c680b1a70 0x7f6c680b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:25.017 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.008+0000 7f6c7038e640 1 -- 172.21.15.40:0/1338166505 >> 172.21.15.40:0/1338166505 conn(0x7f6c68071680 msgr2=0x7f6c680659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:25.017 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.008+0000 7f6c7038e640 1 -- 172.21.15.40:0/1338166505 shutdown_connections 2023-07-29T13:27:25.017 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.008+0000 7f6c7038e640 1 -- 172.21.15.40:0/1338166505 wait complete. 2023-07-29T13:27:25.017 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.009+0000 7f6c7038e640 10 monclient: build_initial_monmap 2023-07-29T13:27:25.017 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.009+0000 7f6c7038e640 10 monclient: monmap: 2023-07-29T13:27:25.017 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:27:25.018 INFO:teuthology.misc.health.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:25.018 INFO:teuthology.misc.health.smithi040.stderr:last_changed 0.000000 2023-07-29T13:27:25.018 INFO:teuthology.misc.health.smithi040.stderr:created 0.000000 2023-07-29T13:27:25.018 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:25.018 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:25.018 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:25.018 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:25.018 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:25.019 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:25.019 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.010+0000 7f6c7038e640 1 librados: starting msgr at 2023-07-29T13:27:25.019 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.010+0000 7f6c7038e640 1 librados: starting objecter 2023-07-29T13:27:25.019 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.010+0000 7f6c7038e640 1 Processor -- start 2023-07-29T13:27:25.019 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.010+0000 7f6c7038e640 1 -- start start 2023-07-29T13:27:25.019 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.010+0000 7f6c7038e640 1 librados: setting wanted keys 2023-07-29T13:27:25.019 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.010+0000 7f6c7038e640 1 librados: calling monclient init 2023-07-29T13:27:25.019 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.010+0000 7f6c7038e640 10 monclient: init 2023-07-29T13:27:25.020 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:25.020 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:27:25.020 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6c680b1a70 0x7f6c68143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:25.020 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 10 monclient(hunting): picked mon.noname-c con 0x7f6c680b1a70 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:25.020 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6813dcb0 0x7f6c681d8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:25.020 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 10 monclient(hunting): picked mon.noname-b con 0x7f6c6813dcb0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:25.020 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68140da0 0x7f6c681d9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:25.021 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 10 monclient(hunting): picked mon.noname-a con 0x7f6c68140da0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:25.021 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:25.021 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c68144940 con 0x7f6c68140da0 2023-07-29T13:27:25.021 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:25.021 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c681d9a10 con 0x7f6c6813dcb0 2023-07-29T13:27:25.021 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:25.021 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c681d58f0 con 0x7f6c680b1a70 2023-07-29T13:27:25.021 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:25.022 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.011+0000 7f6c7038e640 10 monclient(hunting): authenticate will time out at 1367.144435s 2023-07-29T13:27:25.022 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e92b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68140da0 0x7f6c681d9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:25.022 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6d929640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6813dcb0 0x7f6c681d8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:25.022 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e12a640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6c680b1a70 0x7f6c68143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:25.022 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e92b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68140da0 0x7f6c681d9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33730/0 (socket says 172.21.15.40:33730) 2023-07-29T13:27:25.022 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e92b640 1 -- 172.21.15.40:0/1395683776 learned_addr learned my addr 172.21.15.40:0/1395683776 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:27:25.022 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e92b640 10 monclient(hunting): get_auth_request con 0x7f6c68140da0 auth_method 0 2023-07-29T13:27:25.022 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e92b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:25.023 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e92b640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:25.023 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e92b640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:25.023 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6d929640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6813dcb0 0x7f6c681d8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:53630/0 (socket says 172.21.15.40:53630) 2023-07-29T13:27:25.023 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6d929640 10 monclient(hunting): get_auth_request con 0x7f6c6813dcb0 auth_method 0 2023-07-29T13:27:25.023 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6d929640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:25.023 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6d929640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:25.023 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6d929640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:25.023 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e12a640 10 monclient(hunting): get_auth_request con 0x7f6c680b1a70 auth_method 0 2023-07-29T13:27:25.024 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e12a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:25.024 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e12a640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:25.024 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.012+0000 7f6c6e12a640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:25.024 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.013+0000 7f6c6e92b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:25.024 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.013+0000 7f6c6e92b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:25.024 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.013+0000 7f6c6e92b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:25.024 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.013+0000 7f6c6d929640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:25.025 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.013+0000 7f6c6d929640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:25.025 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.013+0000 7f6c6d929640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:25.025 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.013+0000 7f6c6e12a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:25.025 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.013+0000 7f6c6e12a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:25.025 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.013+0000 7f6c6e12a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:25.026 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.017+0000 7f6c6e92b640 10 monclient(hunting): handle_auth_done global_id 4167 payload 824 2023-07-29T13:27:25.026 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.017+0000 7f6c6e92b640 1 -- 172.21.15.40:0/1395683776 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6c680b1a70 msgr2=0x7f6c68143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:25.026 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.017+0000 7f6c6e92b640 1 --2- 172.21.15.40:0/1395683776 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6c680b1a70 0x7f6c68143c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:25.026 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.017+0000 7f6c6e92b640 1 -- 172.21.15.40:0/1395683776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6813dcb0 msgr2=0x7f6c681d8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:25.026 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.017+0000 7f6c6e92b640 1 --2- 172.21.15.40:0/1395683776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6813dcb0 0x7f6c681d8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:25.026 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.017+0000 7f6c6e92b640 10 monclient: _finish_hunting 0 2023-07-29T13:27:25.027 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.017+0000 7f6c6e92b640 1 monclient: found mon.noname-a 2023-07-29T13:27:25.027 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.017+0000 7f6c6e92b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:25.027 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.017+0000 7f6c6e92b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:27:25.027 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.017+0000 7f6c6e92b640 1 -- 172.21.15.40:0/1395683776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c58009400 con 0x7f6c68140da0 2023-07-29T13:27:25.027 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.022+0000 7f6c6e92b640 1 --2- 172.21.15.40:0/1395683776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68140da0 0x7f6c681d9390 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000ca80 tx=0x7f6c6000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:25.027 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.022+0000 7f6c5effd640 10 client.?.objecter ms_handle_connect 0x7f6c68140da0 2023-07-29T13:27:25.027 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.022+0000 7f6c5effd640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:25.029 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 1 -- 172.21.15.40:0/1395683776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000ac30 con 0x7f6c68140da0 2023-07-29T13:27:25.029 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:25.029 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:27:25.029 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 10 monclient: dump: 2023-07-29T13:27:25.029 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:25.030 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:25.030 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:25.030 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:25.030 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:25.030 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:25.030 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:25.030 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:25.030 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:25.031 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:25.031 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 10 monclient: _finish_auth 0 2023-07-29T13:27:25.031 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 10 monclient: _check_auth_tickets 2023-07-29T13:27:25.031 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:25.031 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 1 -- 172.21.15.40:0/1395683776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6000ad90 con 0x7f6c68140da0 2023-07-29T13:27:25.031 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:25.031 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 1 -- 172.21.15.40:0/1395683776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60012600 con 0x7f6c68140da0 2023-07-29T13:27:25.031 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:25.032 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:27:25.032 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c5effd640 10 monclient: dump: 2023-07-29T13:27:25.032 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:25.032 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:25.032 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:25.032 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:25.032 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:25.033 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:25.033 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:25.033 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:25.033 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:25.033 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:25.034 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c7038e640 5 monclient: authenticate success, global_id 4167 2023-07-29T13:27:25.034 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c7038e640 10 monclient: _renew_subs 2023-07-29T13:27:25.034 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c7038e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:25.034 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c7038e640 1 -- 172.21.15.40:0/1395683776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c681d5ab0 con 0x7f6c68140da0 2023-07-29T13:27:25.034 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c6c927640 4 set_mon_vals no callback set 2023-07-29T13:27:25.034 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c7038e640 10 client.4167.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:25.035 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c7038e640 10 monclient: _renew_subs 2023-07-29T13:27:25.035 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c7038e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:25.035 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c7038e640 1 -- 172.21.15.40:0/1395683776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c681d5f40 con 0x7f6c68140da0 2023-07-29T13:27:25.035 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.027+0000 7f6c7038e640 1 librados: init done 2023-07-29T13:27:25.035 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.029+0000 7f6c7038e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:25.035 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.029+0000 7f6c7038e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:25.035 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.029+0000 7f6c7038e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:25.035 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.029+0000 7f6c7038e640 1 -- 172.21.15.40:0/1395683776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c681418b0 con 0x7f6c68140da0 2023-07-29T13:27:25.242 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.241+0000 7f6c5effd640 1 -- 172.21.15.40:0/1395683776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 3) v1 ==== 87920+0+0 (secure 0 0 0) 0x7f6c60007070 con 0x7f6c68140da0 2023-07-29T13:27:25.257 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.255+0000 7f6c5effd640 1 -- 172.21.15.40:0/1395683776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f6c6005bd90 con 0x7f6c68140da0 2023-07-29T13:27:25.257 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.255+0000 7f6c5effd640 10 client.4167.objecter ms_dispatch 0x7f6c68000b90 osd_map(10..10 src has 1..10) v4 2023-07-29T13:27:25.258 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.255+0000 7f6c5effd640 3 client.4167.objecter handle_osd_map got epochs [10,10] > 0 2023-07-29T13:27:25.258 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.255+0000 7f6c5effd640 3 client.4167.objecter handle_osd_map decoding full epoch 10 2023-07-29T13:27:25.258 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.255+0000 7f6c5effd640 20 client.4167.objecter dump_active .. 0 homeless 2023-07-29T13:27:25.258 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.255+0000 7f6c5effd640 1 -- 172.21.15.40:0/1395683776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f6c600541c0 con 0x7f6c68140da0 2023-07-29T13:27:25.258 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.256+0000 7f6c5effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:25.258 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.256+0000 7f6c5effd640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:25.304 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:25.303+0000 7f403e455040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-29T13:27:25.387 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.384+0000 7f6c7038e640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:25.388 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.384+0000 7f6c7038e640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:25.388 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.384+0000 7f6c7038e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:27:25.388 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.385+0000 7f6c7038e640 1 -- 172.21.15.40:0/1395683776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c681d9b50 con 0x7f6c68140da0 2023-07-29T13:27:25.396 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:25.394+0000 7f403e455040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-29T13:27:25.461 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.459+0000 7f6c5effd640 1 -- 172.21.15.40:0/1395683776 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f6c60017090 con 0x7f6c68140da0 2023-07-29T13:27:25.462 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.459+0000 7f6c5effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:25.462 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.459+0000 7f6c5effd640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:25.462 INFO:teuthology.misc.health.smithi040.stdout: 2023-07-29T13:27:25.463 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":10,"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","created":"2023-07-29T13:27:17.185104+0000","modified":"2023-07-29T13:27:22.831496+0000","last_up_change":"0.000000","last_in_change":"2023-07-29T13:27:22.831496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6069a944-26b4-4b00-94e2-f91c485e8957","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab18edd4-e1cb-4815-b5a8-af2b4b2e790b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4575bc2c-4d43-46d3-95ba-1dc2c94278e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e7224640-7d33-4bb7-81a5-99fc26da7763","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23cd76f6-788b-48cf-9438-e931659d3a1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0ddb1e3f-0f65-40d6-ab50-5dc521774445","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4b9d2830-9f2c-438c-9c0b-b6692e21f921","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6593807d-34a1-431b-aa23-24e8340d887b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-29T13:27:25.464 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.460+0000 7f6c7038e640 10 librados: watch_flush enter 2023-07-29T13:27:25.465 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.460+0000 7f6c7038e640 10 librados: watch_flush exit 2023-07-29T13:27:25.465 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.460+0000 7f6c7038e640 20 client.4167.objecter shutdown clearing up homeless session... 2023-07-29T13:27:25.465 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.460+0000 7f6c7038e640 10 client.4167.objecter successfully canceled tick 2023-07-29T13:27:25.465 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.460+0000 7f6c7038e640 10 monclient: shutdown 2023-07-29T13:27:25.466 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.461+0000 7f6c7038e640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:25.466 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.461+0000 7f6c7038e640 1 -- 172.21.15.40:0/1395683776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68140da0 msgr2=0x7f6c681d9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:25.466 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.461+0000 7f6c7038e640 1 --2- 172.21.15.40:0/1395683776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68140da0 0x7f6c681d9390 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000ca80 tx=0x7f6c6000cf40 comp rx=0 tx=0).stop 2023-07-29T13:27:25.466 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.461+0000 7f6c7038e640 1 -- 172.21.15.40:0/1395683776 shutdown_connections 2023-07-29T13:27:25.467 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.461+0000 7f6c7038e640 1 --2- 172.21.15.40:0/1395683776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6813dcb0 0x7f6c681d8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:25.467 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.461+0000 7f6c7038e640 1 --2- 172.21.15.40:0/1395683776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68140da0 0x7f6c681d9390 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:25.467 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.461+0000 7f6c7038e640 1 --2- 172.21.15.40:0/1395683776 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6c680b1a70 0x7f6c68143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:25.468 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.461+0000 7f6c7038e640 1 -- 172.21.15.40:0/1395683776 >> 172.21.15.40:0/1395683776 conn(0x7f6c68071680 msgr2=0x7f6c680b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:25.468 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.461+0000 7f6c7038e640 1 -- 172.21.15.40:0/1395683776 shutdown_connections 2023-07-29T13:27:25.468 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.461+0000 7f6c7038e640 1 -- 172.21.15.40:0/1395683776 wait complete. 2023-07-29T13:27:25.468 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:25.461+0000 7f6c7038e640 1 librados: shutdown 2023-07-29T13:27:25.470 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:25.469+0000 7f403e455040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-29T13:27:25.475 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-29T13:27:25.656 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:25.654+0000 7f403e455040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-29T13:27:25.913 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:25.912+0000 7f403e455040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-29T13:27:26.102 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:26.101+0000 7f403e455040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-29T13:27:26.190 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:26.184+0000 7f403e455040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-29T13:27:26.426 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:26.425+0000 7f403e455040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-29T13:27:26.988 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:26.987+0000 7f403e455040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-29T13:27:27.348 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:27.346+0000 7f403e455040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-29T13:27:27.433 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:27.432+0000 7f403e455040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-29T13:27:27.507 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:27.506+0000 7f403e455040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-29T13:27:27.583 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:27.582+0000 7f403e455040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-29T13:27:27.974 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:27.973+0000 7f403e455040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-29T13:27:28.060 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:28.059+0000 7f403e455040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-29T13:27:28.136 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:28.134+0000 7f403e455040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-29T13:27:28.384 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:28.383+0000 7f403e455040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-29T13:27:28.455 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:28.454+0000 7f403e455040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-29T13:27:28.531 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:28.530+0000 7f403e455040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-29T13:27:28.602 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:28.601+0000 7f403e455040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-29T13:27:30.882 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:30.881+0000 7f3fe02d5640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-29T13:27:31.053 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T13:27:31.052+0000 7f3fe02d5640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-29T13:27:31.773 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:08.448 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:31.806 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:08.482 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:31.833 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:08.487 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:31.848 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:08.523 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:31.895 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:08.572 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:31.900 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:08.555 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.006 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:08.689 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.073 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:08.758 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.074 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:08.757 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.148 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:08.832 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.152 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:08.831 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.219 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:08.899 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.254 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:08.939 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.281 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-29T13:27:32.314 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:09.000 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.374 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.371+0000 7f32c5cbe640 10 monclient: get_monmap_and_config 2023-07-29T13:27:32.374 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.371+0000 7f32c5cbe640 10 monclient: build_initial_monmap 2023-07-29T13:27:32.375 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.371+0000 7f32c5cbe640 10 monclient: monmap: 2023-07-29T13:27:32.375 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:27:32.375 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:32.376 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:27:32.373012+0000 2023-07-29T13:27:32.376 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:27:32.373012+0000 2023-07-29T13:27:32.376 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:32.376 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:32.377 INFO:teuthology.misc.health.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:32.377 INFO:teuthology.misc.health.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:32.377 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:32.377 INFO:teuthology.misc.health.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:32.377 INFO:teuthology.misc.health.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:32.378 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:32.379 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:27:32.370+0000 5dc1300 -1 Falling back to public interface 2023-07-29T13:27:32.380 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.372+0000 7f32c5cbe640 1 Processor -- start 2023-07-29T13:27:32.380 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.372+0000 7f32c5cbe640 1 -- start start 2023-07-29T13:27:32.381 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.372+0000 7f32c5cbe640 10 monclient: init 2023-07-29T13:27:32.381 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:32.381 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 10 monclient: _add_conns ranks=[3,0,1,2,4] 2023-07-29T13:27:32.381 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f32c00b1a70 0x7f32c00b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:32.381 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 10 monclient(hunting): picked mon.noname-a con 0x7f32c00b1a70 addr v2:172.21.15.40:3300/0 2023-07-29T13:27:32.381 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f32c013dcb0 addr v1:172.21.15.40:6789/0 2023-07-29T13:27:32.381 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f32c0140da0 addr v1:172.21.15.45:6789/0 2023-07-29T13:27:32.382 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c00a9740 con 0x7f32c013dcb0 2023-07-29T13:27:32.382 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c00a9a20 con 0x7f32c0140da0 2023-07-29T13:27:32.382 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:32.382 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c00a9540 con 0x7f32c00b1a70 2023-07-29T13:27:32.383 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:32.383 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.373+0000 7f32c5cbe640 10 monclient(hunting): authenticate will time out at 1374.506452s 2023-07-29T13:27:32.383 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.374+0000 7f32bffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f32c0140da0 0x7f32c0141200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.40:57100/0 (socket says 172.21.15.40:57100) 2023-07-29T13:27:32.383 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.374+0000 7f32bffff640 1 -- 172.21.15.40:0/1806170453 learned_addr learned my addr 172.21.15.40:0/1806170453 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:27:32.383 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.377+0000 7f32bf7fe640 1 --2- 172.21.15.40:0/1806170453 >> v2:172.21.15.40:3300/0 conn(0x7f32c00b1a70 0x7f32c00b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:32.383 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.378+0000 7f32bf7fe640 10 monclient(hunting): get_auth_request con 0x7f32c00b1a70 auth_method 0 2023-07-29T13:27:32.383 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.378+0000 7f32bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:32.383 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.378+0000 7f32bf7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:32.384 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.378+0000 7f32bf7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:32.384 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.379+0000 7f32bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:32.384 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.379+0000 7f32bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:32.384 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.379+0000 7f32bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:32.385 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.381+0000 7f32bf7fe640 10 monclient(hunting): handle_auth_done global_id 4212 payload 274 2023-07-29T13:27:32.385 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.381+0000 7f32bf7fe640 1 -- 172.21.15.40:0/1806170453 >> v1:172.21.15.45:6789/0 conn(0x7f32c0140da0 legacy=0x7f32c0141200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:32.385 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.381+0000 7f32bf7fe640 1 -- 172.21.15.40:0/1806170453 >> v1:172.21.15.40:6789/0 conn(0x7f32c013dcb0 legacy=0x7f32c013e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:32.385 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.381+0000 7f32bf7fe640 10 monclient: _finish_hunting 0 2023-07-29T13:27:32.385 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.381+0000 7f32bf7fe640 1 monclient: found mon.noname-a 2023-07-29T13:27:32.386 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.381+0000 7f32bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:32.386 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.381+0000 7f32bf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:27:32.386 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.381+0000 7f32bf7fe640 1 -- 172.21.15.40:0/1806170453 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c00a9d00 con 0x7f32c00b1a70 2023-07-29T13:27:32.386 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.383+0000 7f32bf7fe640 1 --2- 172.21.15.40:0/1806170453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32c00b1a70 0x7f32c00b1e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f32b00097a0 tx=0x7f32b002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cdc9368573fa1 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:32.435 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 1 -- 172.21.15.40:0/1806170453 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b002e070 con 0x7f32c00b1a70 2023-07-29T13:27:32.436 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:32.436 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:27:32.436 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 10 monclient: dump: 2023-07-29T13:27:32.437 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:32.437 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:32.437 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:32.437 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:32.438 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:32.438 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:32.438 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:32.439 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:32.439 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:32.439 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:32.439 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 10 monclient: _finish_auth 0 2023-07-29T13:27:32.440 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 10 monclient: _check_auth_tickets 2023-07-29T13:27:32.440 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:32.440 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 1 -- 172.21.15.40:0/1806170453 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b00315a0 con 0x7f32c00b1a70 2023-07-29T13:27:32.440 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:32.440 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 1 -- 172.21.15.40:0/1806170453 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0031930 con 0x7f32c00b1a70 2023-07-29T13:27:32.441 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32c5cbe640 5 monclient: authenticate success, global_id 4212 2023-07-29T13:27:32.441 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32c5cbe640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:32.441 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32c5cbe640 4 set_mon_vals no callback set 2023-07-29T13:27:32.441 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:32.441 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:27:32.441 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.432+0000 7f32be7fc640 10 monclient: dump: 2023-07-29T13:27:32.442 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:32.442 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:32.442 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:32.442 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:32.442 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:32.442 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:32.443 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:32.443 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:32.443 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:32.443 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:32.443 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.433+0000 7f32c5cbe640 10 monclient: shutdown 2023-07-29T13:27:32.443 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.433+0000 7f32c5cbe640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:32.444 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.433+0000 7f32c5cbe640 1 -- 172.21.15.40:0/1806170453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32c00b1a70 msgr2=0x7f32c00b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:32.444 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.433+0000 7f32c5cbe640 1 --2- 172.21.15.40:0/1806170453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32c00b1a70 0x7f32c00b1e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f32b00097a0 tx=0x7f32b002b840 comp rx=0 tx=0).stop 2023-07-29T13:27:32.444 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.433+0000 7f32c5cbe640 1 -- 172.21.15.40:0/1806170453 shutdown_connections 2023-07-29T13:27:32.444 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.433+0000 7f32c5cbe640 1 --2- 172.21.15.40:0/1806170453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32c00b1a70 0x7f32c00b1e40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:32.444 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.433+0000 7f32c5cbe640 1 -- 172.21.15.40:0/1806170453 >> 172.21.15.40:0/1806170453 conn(0x7f32c0071680 msgr2=0x7f32c00659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:32.445 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.433+0000 7f32c5cbe640 1 -- 172.21.15.40:0/1806170453 shutdown_connections 2023-07-29T13:27:32.445 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.433+0000 7f32c5cbe640 1 -- 172.21.15.40:0/1806170453 wait complete. 2023-07-29T13:27:32.449 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.434+0000 7f32c5cbe640 10 monclient: build_initial_monmap 2023-07-29T13:27:32.450 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.434+0000 7f32c5cbe640 10 monclient: monmap: 2023-07-29T13:27:32.450 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:27:32.450 INFO:teuthology.misc.health.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:32.450 INFO:teuthology.misc.health.smithi040.stderr:last_changed 0.000000 2023-07-29T13:27:32.450 INFO:teuthology.misc.health.smithi040.stderr:created 0.000000 2023-07-29T13:27:32.451 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:32.451 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:32.451 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:32.451 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:32.452 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:32.452 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:32.452 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.434+0000 7f32c5cbe640 1 librados: starting msgr at 2023-07-29T13:27:32.452 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.434+0000 7f32c5cbe640 1 librados: starting objecter 2023-07-29T13:27:32.453 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.434+0000 7f32c5cbe640 1 Processor -- start 2023-07-29T13:27:32.453 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 1 -- start start 2023-07-29T13:27:32.453 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 1 librados: setting wanted keys 2023-07-29T13:27:32.454 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 1 librados: calling monclient init 2023-07-29T13:27:32.454 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 10 monclient: init 2023-07-29T13:27:32.454 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:32.454 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T13:27:32.455 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f32c00b1a70 0x7f32c013a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:32.455 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 10 monclient(hunting): picked mon.noname-c con 0x7f32c00b1a70 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:32.455 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32c013dcb0 0x7f32c01d97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:32.455 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 10 monclient(hunting): picked mon.noname-a con 0x7f32c013dcb0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:32.456 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0140da0 0x7f32c00aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:32.456 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 10 monclient(hunting): picked mon.noname-b con 0x7f32c0140da0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:32.456 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:32.457 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c00aa4d0 con 0x7f32c013dcb0 2023-07-29T13:27:32.457 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:32.457 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c01d9290 con 0x7f32c0140da0 2023-07-29T13:27:32.457 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:32.458 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c013c660 con 0x7f32c00b1a70 2023-07-29T13:27:32.458 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:32.458 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.435+0000 7f32c5cbe640 10 monclient(hunting): authenticate will time out at 1374.568757s 2023-07-29T13:27:32.458 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bf7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f32c00b1a70 0x7f32c013a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:32.459 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0140da0 0x7f32c00aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:32.459 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bf7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f32c00b1a70 0x7f32c013a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:55598/0 (socket says 172.21.15.40:55598) 2023-07-29T13:27:32.459 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bf7fe640 1 -- 172.21.15.40:0/3316964991 learned_addr learned my addr 172.21.15.40:0/3316964991 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:27:32.460 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bf7fe640 10 monclient(hunting): get_auth_request con 0x7f32c00b1a70 auth_method 0 2023-07-29T13:27:32.460 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:32.460 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bf7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:32.460 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bf7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:32.461 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bffff640 10 monclient(hunting): get_auth_request con 0x7f32c0140da0 auth_method 0 2023-07-29T13:27:32.461 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:32.461 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bffff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:32.461 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.436+0000 7f32bffff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:32.462 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.437+0000 7f32bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:32.462 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.437+0000 7f32bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:32.462 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.437+0000 7f32bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:32.462 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.437+0000 7f32bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:32.463 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.437+0000 7f32bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:32.463 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.437+0000 7f32bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:32.463 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.437+0000 7f32beffd640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32c013dcb0 0x7f32c01d97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:32.463 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.438+0000 7f32beffd640 10 monclient(hunting): get_auth_request con 0x7f32c013dcb0 auth_method 0 2023-07-29T13:27:32.464 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.438+0000 7f32beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:32.464 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.438+0000 7f32beffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:32.467 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.438+0000 7f32beffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:32.468 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.439+0000 7f32beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:32.468 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.439+0000 7f32beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:32.468 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.439+0000 7f32beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:32.468 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.439+0000 7f32bf7fe640 10 monclient(hunting): handle_auth_done global_id 4151 payload 824 2023-07-29T13:27:32.468 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.439+0000 7f32bf7fe640 1 -- 172.21.15.40:0/3316964991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0140da0 msgr2=0x7f32c00aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:32.469 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.439+0000 7f32bf7fe640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0140da0 0x7f32c00aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:32.469 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.439+0000 7f32bf7fe640 1 -- 172.21.15.40:0/3316964991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32c013dcb0 msgr2=0x7f32c01d97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:32.469 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.439+0000 7f32bf7fe640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32c013dcb0 0x7f32c01d97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:32.469 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.439+0000 7f32bf7fe640 10 monclient: _finish_hunting 0 2023-07-29T13:27:32.469 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.439+0000 7f32bf7fe640 1 monclient: found mon.noname-c 2023-07-29T13:27:32.469 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.440+0000 7f32bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:32.469 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.440+0000 7f32bf7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:27:32.470 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.440+0000 7f32bf7fe640 1 -- 172.21.15.40:0/3316964991 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b0009400 con 0x7f32c00b1a70 2023-07-29T13:27:32.470 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.442+0000 7f32bf7fe640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f32c00b1a70 0x7f32c013a870 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f32b0034040 tx=0x7f32b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:32.470 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.442+0000 7f32a7fff640 10 client.?.objecter ms_handle_connect 0x7f32c00b1a70 2023-07-29T13:27:32.470 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.442+0000 7f32a7fff640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:32.470 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 1 -- 172.21.15.40:0/3316964991 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0032900 con 0x7f32c00b1a70 2023-07-29T13:27:32.470 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:32.470 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:27:32.471 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 10 monclient: dump: 2023-07-29T13:27:32.471 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:32.471 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:32.471 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:32.471 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:32.471 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:32.471 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:32.471 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:32.472 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:32.472 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:32.472 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:32.472 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 10 monclient: _finish_auth 0 2023-07-29T13:27:32.472 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 10 monclient: _check_auth_tickets 2023-07-29T13:27:32.472 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:32.472 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 1 -- 172.21.15.40:0/3316964991 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b0032aa0 con 0x7f32c00b1a70 2023-07-29T13:27:32.473 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32c5cbe640 5 monclient: authenticate success, global_id 4151 2023-07-29T13:27:32.473 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32c5cbe640 10 monclient: _renew_subs 2023-07-29T13:27:32.473 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32c5cbe640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:32.473 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32c5cbe640 1 -- 172.21.15.40:0/3316964991 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c01d9cf0 con 0x7f32c00b1a70 2023-07-29T13:27:32.473 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:32.473 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 1 -- 172.21.15.40:0/3316964991 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0032e50 con 0x7f32c00b1a70 2023-07-29T13:27:32.473 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:32.473 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:27:32.474 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32a7fff640 10 monclient: dump: 2023-07-29T13:27:32.474 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:32.474 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:32.474 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:32.474 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:32.474 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:32.474 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:32.474 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:32.475 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:32.475 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:32.475 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:32.475 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32bdffb640 4 set_mon_vals no callback set 2023-07-29T13:27:32.475 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32c5cbe640 10 client.4151.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:32.475 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32c5cbe640 10 monclient: _renew_subs 2023-07-29T13:27:32.475 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32c5cbe640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:32.475 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32c5cbe640 1 -- 172.21.15.40:0/3316964991 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c00ab410 con 0x7f32c00b1a70 2023-07-29T13:27:32.476 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.446+0000 7f32c5cbe640 1 librados: init done 2023-07-29T13:27:32.476 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.447+0000 7f32c5cbe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:32.476 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.447+0000 7f32c5cbe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:32.476 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.447+0000 7f32c5cbe640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:32.476 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.447+0000 7f32c5cbe640 1 -- 172.21.15.40:0/3316964991 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c00ab660 con 0x7f32c00b1a70 2023-07-29T13:27:32.477 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:27:32.457+0000 5dc1300 -1 Falling back to public interface 2023-07-29T13:27:32.477 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:27:32.459+0000 5dc1300 -1 Falling back to public interface 2023-07-29T13:27:32.478 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.467+0000 7f32a7fff640 1 -- 172.21.15.40:0/3316964991 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 6) v1 ==== 88153+0+0 (secure 0 0 0) 0x7f32b004e430 con 0x7f32c00b1a70 2023-07-29T13:27:32.478 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.467+0000 7f32a7fff640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f32ac06bf20 0x7f32ac06e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:32.478 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.467+0000 7f32beffd640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f32ac06bf20 0x7f32ac06e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:32.478 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.467+0000 7f32beffd640 10 monclient: get_auth_request con 0x7f32ac06bf20 auth_method 0 2023-07-29T13:27:32.478 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.468+0000 7f32a7fff640 10 client.4151.objecter ms_handle_connect 0x7f32ac06bf20 2023-07-29T13:27:32.478 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.468+0000 7f32beffd640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f32ac06bf20 0x7f32ac06e3d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f32c01d6a70 tx=0x7f32b4009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:32.623 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:27:32.612+0000 5dc1300 -1 Falling back to public interface 2023-07-29T13:27:32.694 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:27:32.686+0000 5dc1300 -1 Falling back to public interface 2023-07-29T13:27:32.753 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.751+0000 7f32a7fff640 1 -- 172.21.15.40:0/3316964991 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f32b007d980 con 0x7f32c00b1a70 2023-07-29T13:27:32.754 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.751+0000 7f32a7fff640 10 client.4151.objecter ms_dispatch 0x7f32c0000b90 osd_map(11..11 src has 1..11) v4 2023-07-29T13:27:32.754 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.751+0000 7f32a7fff640 3 client.4151.objecter handle_osd_map got epochs [11,11] > 0 2023-07-29T13:27:32.754 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.751+0000 7f32a7fff640 3 client.4151.objecter handle_osd_map decoding full epoch 11 2023-07-29T13:27:32.755 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.751+0000 7f32a7fff640 20 client.4151.objecter dump_active .. 0 homeless 2023-07-29T13:27:32.755 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.752+0000 7f32a7fff640 1 -- 172.21.15.40:0/3316964991 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f32b0075db0 con 0x7f32c00b1a70 2023-07-29T13:27:32.755 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.752+0000 7f32a7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:32.756 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.752+0000 7f32a7fff640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:32.780 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:27:32.771+0000 5dc1300 -1 Falling back to public interface 2023-07-29T13:27:32.855 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:09.529 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.901 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:27:32.892+0000 5dc1300 -1 Falling back to public interface 2023-07-29T13:27:32.941 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:09.618 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.971 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:09.625 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:32.987 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.985+0000 7f32c5cbe640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:32.987 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.985+0000 7f32c5cbe640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:32.988 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.985+0000 7f32c5cbe640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:32.988 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:32.985+0000 7f32c5cbe640 1 -- 172.21.15.40:0/3316964991 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32c01d9090 con 0x7f32c00b1a70 2023-07-29T13:27:33.035 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.034+0000 7f32a7fff640 1 -- 172.21.15.40:0/3316964991 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f32b00793d0 con 0x7f32c00b1a70 2023-07-29T13:27:33.036 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.034+0000 7f32a7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:33.036 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.034+0000 7f32a7fff640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:33.036 INFO:teuthology.misc.health.smithi040.stdout: 2023-07-29T13:27:33.036 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":11,"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","created":"2023-07-29T13:27:17.185104+0000","modified":"2023-07-29T13:27:28.649513+0000","last_up_change":"0.000000","last_in_change":"2023-07-29T13:27:22.831496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6069a944-26b4-4b00-94e2-f91c485e8957","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab18edd4-e1cb-4815-b5a8-af2b4b2e790b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4575bc2c-4d43-46d3-95ba-1dc2c94278e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e7224640-7d33-4bb7-81a5-99fc26da7763","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23cd76f6-788b-48cf-9438-e931659d3a1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0ddb1e3f-0f65-40d6-ab50-5dc521774445","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4b9d2830-9f2c-438c-9c0b-b6692e21f921","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6593807d-34a1-431b-aa23-24e8340d887b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-29T13:27:33.038 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 10 librados: watch_flush enter 2023-07-29T13:27:33.038 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 10 librados: watch_flush exit 2023-07-29T13:27:33.038 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 20 client.4151.objecter shutdown clearing up homeless session... 2023-07-29T13:27:33.039 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 10 client.4151.objecter successfully canceled tick 2023-07-29T13:27:33.039 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 1 -- 172.21.15.40:0/3316964991 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f32ac06bf20 msgr2=0x7f32ac06e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:33.039 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f32ac06bf20 0x7f32ac06e3d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f32c01d6a70 tx=0x7f32b4009040 comp rx=0 tx=0).stop 2023-07-29T13:27:33.039 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 10 monclient: shutdown 2023-07-29T13:27:33.039 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:33.039 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 1 -- 172.21.15.40:0/3316964991 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f32c00b1a70 msgr2=0x7f32c013a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:33.039 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f32c00b1a70 0x7f32c013a870 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f32b0034040 tx=0x7f32b00046c0 comp rx=0 tx=0).stop 2023-07-29T13:27:33.040 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 1 -- 172.21.15.40:0/3316964991 shutdown_connections 2023-07-29T13:27:33.040 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f32ac06bf20 0x7f32ac06e3d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:33.040 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0140da0 0x7f32c00aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:33.040 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.036+0000 7f32c5cbe640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32c013dcb0 0x7f32c01d97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:33.040 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.037+0000 7f32c5cbe640 1 --2- 172.21.15.40:0/3316964991 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f32c00b1a70 0x7f32c013a870 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:33.040 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.037+0000 7f32c5cbe640 1 -- 172.21.15.40:0/3316964991 >> 172.21.15.40:0/3316964991 conn(0x7f32c0071680 msgr2=0x7f32c00b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:33.040 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.037+0000 7f32c5cbe640 1 -- 172.21.15.40:0/3316964991 shutdown_connections 2023-07-29T13:27:33.041 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.037+0000 7f32c5cbe640 1 -- 172.21.15.40:0/3316964991 wait complete. 2023-07-29T13:27:33.041 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:33.037+0000 7f32c5cbe640 1 librados: shutdown 2023-07-29T13:27:33.049 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-29T13:27:33.170 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:09.853 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.205 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:09.880 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.207 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:09.891 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.289 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:09.966 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.299 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:09.978 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.338 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:09.992 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.444 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:10.130 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.520 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:10.204 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.561 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:10.246 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.570 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:10.245 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.648 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:10.327 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.674 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:10.351 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.685 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:10.339 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.705 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:10.366 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.766 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:10.427 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.795 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:10.480 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.859 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:10.534 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.881 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:10.564 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.921 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:10.605 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.966 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:10.643 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:33.986 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:10.641 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.016 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:10.696 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.148 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:10.823 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.160 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:10.846 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.173 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:10.856 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.213 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:10.897 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.256 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:10.933 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.292 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:10.947 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.306 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:10.985 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.357 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:27:34.350+0000 5dc1300 -1 Falling back to public interface 2023-07-29T13:27:34.442 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:11.116 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.450 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:11.136 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.472 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:11.156 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.510 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:11.194 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.546 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:11.223 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.586 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:11.241 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.599 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:11.278 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.730 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:11.405 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.743 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:11.428 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.768 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:11.451 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.777 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:11.452 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.810 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:11.494 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.830 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:11.491 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.836 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:11.513 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.875 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:11.530 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:34.889 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:11.568 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.032 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:11.718 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.059 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:11.743 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.066 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:11.741 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.103 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:11.787 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.135 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:11.812 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.176 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:11.831 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.177 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:11.857 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.186 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:11.847 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.324 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:12.009 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.353 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:12.036 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.396 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:12.080 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.427 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:12.104 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.466 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:12.121 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.471 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:12.150 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.562 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:12.223 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.613 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:12.298 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.617 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:12.292 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.646 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:12.329 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.673 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:12.348 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.686 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:12.370 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.760 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:12.440 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.862 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:12.523 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.909 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:12.594 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.957 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:12.634 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:35.998 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:12.652 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.014 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:12.690 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.054 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:12.709 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.151 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:12.812 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.174 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:12.857 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.218 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:12.903 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.256 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:12.939 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.275 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:12.959 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.290 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:12.969 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.348 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:13.027 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.463 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:13.124 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.481 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:13.167 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.552 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:13.237 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:36.760 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:13.421 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:37.061 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:13.723 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:37.328 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:13.989 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:37.845 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:14.507 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:37.902 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:14.563 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:39.854 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-29T13:27:39.961 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.959+0000 7f8c7ea41640 10 monclient: get_monmap_and_config 2023-07-29T13:27:39.961 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.959+0000 7f8c7ea41640 10 monclient: build_initial_monmap 2023-07-29T13:27:39.961 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.959+0000 7f8c7ea41640 10 monclient: monmap: 2023-07-29T13:27:39.961 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:27:39.961 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:39.962 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:27:39.960470+0000 2023-07-29T13:27:39.962 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:27:39.960470+0000 2023-07-29T13:27:39.962 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:39.962 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:39.962 INFO:teuthology.misc.health.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:39.962 INFO:teuthology.misc.health.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:39.962 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:39.962 INFO:teuthology.misc.health.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:39.963 INFO:teuthology.misc.health.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:39.963 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:39.964 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.960+0000 7f8c7ea41640 1 Processor -- start 2023-07-29T13:27:39.964 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.960+0000 7f8c7ea41640 1 -- start start 2023-07-29T13:27:39.964 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.960+0000 7f8c7ea41640 10 monclient: init 2023-07-29T13:27:39.964 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.960+0000 7f8c7ea41640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:39.964 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.960+0000 7f8c7ea41640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2023-07-29T13:27:39.965 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.960+0000 7f8c7ea41640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c780b1a70 0x7f8c780b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:39.965 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.960+0000 7f8c7ea41640 10 monclient(hunting): picked mon.noname-b con 0x7f8c780b1a70 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:39.965 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.960+0000 7f8c7ea41640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c7813dcb0 0x7f8c7813e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:39.965 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.960+0000 7f8c7ea41640 10 monclient(hunting): picked mon.noname-a con 0x7f8c7813dcb0 addr v2:172.21.15.40:3300/0 2023-07-29T13:27:39.965 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.961+0000 7f8c7ea41640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8c78140da0 addr v1:172.21.15.40:6789/0 2023-07-29T13:27:39.965 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.961+0000 7f8c7ea41640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7813e620 con 0x7f8c78140da0 2023-07-29T13:27:39.965 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.961+0000 7f8c7ea41640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:39.966 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.961+0000 7f8c7ea41640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c780a9540 con 0x7f8c7813dcb0 2023-07-29T13:27:39.966 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.961+0000 7f8c7ea41640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:39.966 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.961+0000 7f8c7ea41640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c780a9740 con 0x7f8c780b1a70 2023-07-29T13:27:39.966 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.961+0000 7f8c7ea41640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:39.966 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.961+0000 7f8c7ea41640 10 monclient(hunting): authenticate will time out at 1382.094284s 2023-07-29T13:27:39.966 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.961+0000 7f8c77fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c780b1a70 0x7f8c780b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:39.966 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.961+0000 7f8c777fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c7813dcb0 0x7f8c7813e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:39.969 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.962+0000 7f8c777fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c7813dcb0 0x7f8c7813e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51668/0 (socket says 172.21.15.40:51668) 2023-07-29T13:27:39.969 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.962+0000 7f8c777fe640 1 -- 172.21.15.40:0/801022944 learned_addr learned my addr 172.21.15.40:0/801022944 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:27:39.969 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.962+0000 7f8c777fe640 10 monclient(hunting): get_auth_request con 0x7f8c7813dcb0 auth_method 0 2023-07-29T13:27:39.969 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.962+0000 7f8c777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:39.969 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.962+0000 7f8c777fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:39.970 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.962+0000 7f8c777fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:39.970 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.963+0000 7f8c777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:39.970 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.963+0000 7f8c777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:39.970 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.963+0000 7f8c777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:39.970 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.965+0000 7f8c777fe640 10 monclient(hunting): handle_auth_done global_id 4221 payload 274 2023-07-29T13:27:39.970 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.965+0000 7f8c777fe640 1 -- 172.21.15.40:0/801022944 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c780b1a70 msgr2=0x7f8c780b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:39.970 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.965+0000 7f8c777fe640 1 --2- 172.21.15.40:0/801022944 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c780b1a70 0x7f8c780b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:39.970 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.965+0000 7f8c777fe640 1 -- 172.21.15.40:0/801022944 >> v1:172.21.15.40:6789/0 conn(0x7f8c78140da0 legacy=0x7f8c781411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:39.971 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.965+0000 7f8c777fe640 10 monclient: _finish_hunting 0 2023-07-29T13:27:39.971 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.965+0000 7f8c777fe640 1 monclient: found mon.noname-a 2023-07-29T13:27:39.971 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.965+0000 7f8c777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:39.971 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.965+0000 7f8c777fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:27:39.971 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.965+0000 7f8c777fe640 1 -- 172.21.15.40:0/801022944 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c780a9a20 con 0x7f8c7813dcb0 2023-07-29T13:27:39.971 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.967+0000 7f8c777fe640 1 --2- 172.21.15.40:0/801022944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7813dcb0 0x7f8c7813e0e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009870 tx=0x7f8c6c034090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281a0f946d822adc server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:39.974 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 1 -- 172.21.15.40:0/801022944 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c034e40 con 0x7f8c7813dcb0 2023-07-29T13:27:39.974 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:39.974 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:27:39.975 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 10 monclient: dump: 2023-07-29T13:27:39.975 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:39.975 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:39.975 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:39.975 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:39.975 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:39.975 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:39.975 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:39.976 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:39.976 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:39.976 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:39.976 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 10 monclient: _finish_auth 0 2023-07-29T13:27:39.976 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 10 monclient: _check_auth_tickets 2023-07-29T13:27:39.976 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:39.976 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 1 -- 172.21.15.40:0/801022944 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6c03c440 con 0x7f8c7813dcb0 2023-07-29T13:27:39.977 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:39.977 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 1 -- 172.21.15.40:0/801022944 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c03c7b0 con 0x7f8c7813dcb0 2023-07-29T13:27:39.977 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:39.977 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:27:39.977 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c76ffd640 10 monclient: dump: 2023-07-29T13:27:39.977 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:39.977 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:39.977 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:39.978 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:39.978 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:39.978 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:39.978 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:39.978 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:39.978 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:39.978 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:39.979 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c7ea41640 5 monclient: authenticate success, global_id 4221 2023-07-29T13:27:39.979 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c7ea41640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:39.979 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.972+0000 7f8c7ea41640 4 set_mon_vals no callback set 2023-07-29T13:27:39.980 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.973+0000 7f8c7ea41640 10 monclient: shutdown 2023-07-29T13:27:39.980 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.973+0000 7f8c7ea41640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:39.981 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.973+0000 7f8c7ea41640 1 -- 172.21.15.40:0/801022944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7813dcb0 msgr2=0x7f8c7813e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:39.981 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.973+0000 7f8c7ea41640 1 --2- 172.21.15.40:0/801022944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7813dcb0 0x7f8c7813e0e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009870 tx=0x7f8c6c034090 comp rx=0 tx=0).stop 2023-07-29T13:27:39.981 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.973+0000 7f8c7ea41640 1 -- 172.21.15.40:0/801022944 shutdown_connections 2023-07-29T13:27:39.981 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.973+0000 7f8c7ea41640 1 --2- 172.21.15.40:0/801022944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7813dcb0 0x7f8c7813e0e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:39.981 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.973+0000 7f8c7ea41640 1 --2- 172.21.15.40:0/801022944 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c780b1a70 0x7f8c780b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:39.981 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.973+0000 7f8c7ea41640 1 -- 172.21.15.40:0/801022944 >> 172.21.15.40:0/801022944 conn(0x7f8c78071680 msgr2=0x7f8c780659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:39.981 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.973+0000 7f8c7ea41640 1 -- 172.21.15.40:0/801022944 shutdown_connections 2023-07-29T13:27:39.982 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.973+0000 7f8c7ea41640 1 -- 172.21.15.40:0/801022944 wait complete. 2023-07-29T13:27:39.982 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.974+0000 7f8c7ea41640 10 monclient: build_initial_monmap 2023-07-29T13:27:39.982 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.974+0000 7f8c7ea41640 10 monclient: monmap: 2023-07-29T13:27:39.982 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:27:39.982 INFO:teuthology.misc.health.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:39.982 INFO:teuthology.misc.health.smithi040.stderr:last_changed 0.000000 2023-07-29T13:27:39.982 INFO:teuthology.misc.health.smithi040.stderr:created 0.000000 2023-07-29T13:27:39.982 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:39.983 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:39.983 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:39.983 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:39.983 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:39.983 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:39.983 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.975+0000 7f8c7ea41640 1 librados: starting msgr at 2023-07-29T13:27:39.983 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.975+0000 7f8c7ea41640 1 librados: starting objecter 2023-07-29T13:27:39.983 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.975+0000 7f8c7ea41640 1 Processor -- start 2023-07-29T13:27:39.984 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.975+0000 7f8c7ea41640 1 -- start start 2023-07-29T13:27:39.984 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.975+0000 7f8c7ea41640 1 librados: setting wanted keys 2023-07-29T13:27:39.984 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.975+0000 7f8c7ea41640 1 librados: calling monclient init 2023-07-29T13:27:39.984 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.975+0000 7f8c7ea41640 10 monclient: init 2023-07-29T13:27:39.984 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:39.984 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T13:27:39.984 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780b1a70 0x7f8c780b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:39.985 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 10 monclient(hunting): picked mon.noname-a con 0x7f8c780b1a70 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:39.985 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7813dcb0 0x7f8c7806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:39.985 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 10 monclient(hunting): picked mon.noname-b con 0x7f8c7813dcb0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:39.985 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c78140da0 0x7f8c781d4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:39.985 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 10 monclient(hunting): picked mon.noname-c con 0x7f8c78140da0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:39.985 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:39.985 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7806ebb0 con 0x7f8c780b1a70 2023-07-29T13:27:39.985 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:39.986 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7806d270 con 0x7f8c7813dcb0 2023-07-29T13:27:39.986 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:39.986 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c780b0000 con 0x7f8c78140da0 2023-07-29T13:27:39.986 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:39.986 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7ea41640 10 monclient(hunting): authenticate will time out at 1382.109298s 2023-07-29T13:27:39.986 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c77fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780b1a70 0x7f8c780b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:39.986 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c7cfde640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c78140da0 0x7f8c781d4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:39.986 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.976+0000 7f8c777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7813dcb0 0x7f8c7806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:39.987 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c7cfde640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c78140da0 0x7f8c781d4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:55606/0 (socket says 172.21.15.40:55606) 2023-07-29T13:27:39.987 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c77fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780b1a70 0x7f8c780b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51678/0 (socket says 172.21.15.40:51678) 2023-07-29T13:27:39.987 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c7cfde640 1 -- 172.21.15.40:0/3104993988 learned_addr learned my addr 172.21.15.40:0/3104993988 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:27:39.987 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c7cfde640 10 monclient(hunting): get_auth_request con 0x7f8c78140da0 auth_method 0 2023-07-29T13:27:39.987 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c7cfde640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:39.987 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c7cfde640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:39.987 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c7cfde640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:39.988 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c77fff640 10 monclient(hunting): get_auth_request con 0x7f8c780b1a70 auth_method 0 2023-07-29T13:27:39.988 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:39.988 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c77fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:39.988 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c77fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:39.988 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c777fe640 10 monclient(hunting): get_auth_request con 0x7f8c7813dcb0 auth_method 0 2023-07-29T13:27:39.988 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:39.988 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c777fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:39.988 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c777fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:39.989 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c7cfde640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:39.989 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c7cfde640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:39.989 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c7cfde640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:39.989 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:39.989 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:39.989 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.977+0000 7f8c77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:39.989 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.978+0000 7f8c777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:39.989 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.978+0000 7f8c777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:39.990 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.978+0000 7f8c777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:39.991 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.980+0000 7f8c7cfde640 10 monclient(hunting): handle_auth_done global_id 4154 payload 824 2023-07-29T13:27:39.991 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.980+0000 7f8c7cfde640 1 -- 172.21.15.40:0/3104993988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7813dcb0 msgr2=0x7f8c7806f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:39.991 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.980+0000 7f8c7cfde640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7813dcb0 0x7f8c7806f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:39.991 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.980+0000 7f8c7cfde640 1 -- 172.21.15.40:0/3104993988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780b1a70 msgr2=0x7f8c780b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:39.991 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.980+0000 7f8c7cfde640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780b1a70 0x7f8c780b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:39.991 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.980+0000 7f8c7cfde640 10 monclient: _finish_hunting 0 2023-07-29T13:27:39.991 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.980+0000 7f8c7cfde640 1 monclient: found mon.noname-c 2023-07-29T13:27:39.992 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.980+0000 7f8c7cfde640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:39.992 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.980+0000 7f8c7cfde640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:27:39.992 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.980+0000 7f8c7cfde640 1 -- 172.21.15.40:0/3104993988 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c0094d0 con 0x7f8c78140da0 2023-07-29T13:27:39.992 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.983+0000 7f8c7cfde640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c78140da0 0x7f8c781d4a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000b990 tx=0x7f8c7000be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:39.992 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.983+0000 7f8c74ff9640 10 client.?.objecter ms_handle_connect 0x7f8c78140da0 2023-07-29T13:27:39.992 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.983+0000 7f8c74ff9640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:39.992 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 1 -- 172.21.15.40:0/3104993988 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7000cba0 con 0x7f8c78140da0 2023-07-29T13:27:39.993 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:39.993 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:27:39.993 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 10 monclient: dump: 2023-07-29T13:27:39.993 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:39.993 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:39.993 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:39.993 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:39.993 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:39.994 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:39.994 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:39.994 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:39.994 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:39.995 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:39.995 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 10 monclient: _finish_auth 0 2023-07-29T13:27:39.995 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 10 monclient: _check_auth_tickets 2023-07-29T13:27:39.995 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:39.996 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 1 -- 172.21.15.40:0/3104993988 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7000cd00 con 0x7f8c78140da0 2023-07-29T13:27:39.996 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:39.996 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 1 -- 172.21.15.40:0/3104993988 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70014680 con 0x7f8c78140da0 2023-07-29T13:27:39.996 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:39.997 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:27:39.997 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c74ff9640 10 monclient: dump: 2023-07-29T13:27:39.997 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:39.998 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:39.998 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:39.998 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:39.998 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:39.999 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:39.999 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:39.999 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:39.999 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:39.999 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:40.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c767fc640 4 set_mon_vals no callback set 2023-07-29T13:27:40.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c7ea41640 5 monclient: authenticate success, global_id 4154 2023-07-29T13:27:40.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c7ea41640 10 monclient: _renew_subs 2023-07-29T13:27:40.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c7ea41640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:40.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c7ea41640 1 -- 172.21.15.40:0/3104993988 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c781d4f70 con 0x7f8c78140da0 2023-07-29T13:27:40.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c7ea41640 10 client.4154.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:40.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c7ea41640 10 monclient: _renew_subs 2023-07-29T13:27:40.000 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c7ea41640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:40.001 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c7ea41640 1 -- 172.21.15.40:0/3104993988 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7806e7e0 con 0x7f8c78140da0 2023-07-29T13:27:40.001 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.987+0000 7f8c7ea41640 1 librados: init done 2023-07-29T13:27:40.001 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.988+0000 7f8c7ea41640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:40.001 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.989+0000 7f8c7ea41640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:40.001 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.989+0000 7f8c7ea41640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:40.001 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:39.989+0000 7f8c7ea41640 1 -- 172.21.15.40:0/3104993988 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c780b01c0 con 0x7f8c78140da0 2023-07-29T13:27:40.010 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.008+0000 7f8c74ff9640 1 -- 172.21.15.40:0/3104993988 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 6) v1 ==== 88153+0+0 (secure 0 0 0) 0x7f8c7001d3b0 con 0x7f8c78140da0 2023-07-29T13:27:40.010 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.008+0000 7f8c74ff9640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f8c6406bed0 0x7f8c6406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:40.010 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.009+0000 7f8c77fff640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f8c6406bed0 0x7f8c6406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:40.011 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.009+0000 7f8c77fff640 10 monclient: get_auth_request con 0x7f8c6406bed0 auth_method 0 2023-07-29T13:27:40.011 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.009+0000 7f8c74ff9640 10 client.4154.objecter ms_handle_connect 0x7f8c6406bed0 2023-07-29T13:27:40.011 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.009+0000 7f8c77fff640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f8c6406bed0 0x7f8c6406e380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800aac0 tx=0x7f8c68008040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:40.173 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.172+0000 7f8c74ff9640 1 -- 172.21.15.40:0/3104993988 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8c7005bd40 con 0x7f8c78140da0 2023-07-29T13:27:40.174 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.172+0000 7f8c74ff9640 10 client.4154.objecter ms_dispatch 0x7f8c78000b90 osd_map(11..11 src has 1..11) v4 2023-07-29T13:27:40.174 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.172+0000 7f8c74ff9640 3 client.4154.objecter handle_osd_map got epochs [11,11] > 0 2023-07-29T13:27:40.174 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.172+0000 7f8c74ff9640 3 client.4154.objecter handle_osd_map decoding full epoch 11 2023-07-29T13:27:40.174 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.172+0000 7f8c74ff9640 20 client.4154.objecter dump_active .. 0 homeless 2023-07-29T13:27:40.275 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.274+0000 7f8c74ff9640 1 -- 172.21.15.40:0/3104993988 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f8c70054170 con 0x7f8c78140da0 2023-07-29T13:27:40.275 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.274+0000 7f8c74ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:40.276 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.274+0000 7f8c74ff9640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:40.573 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.571+0000 7f8c7ea41640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:40.573 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.571+0000 7f8c7ea41640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:40.573 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.571+0000 7f8c7ea41640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:40.573 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.571+0000 7f8c7ea41640 1 -- 172.21.15.40:0/3104993988 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c780b0140 con 0x7f8c78140da0 2023-07-29T13:27:40.600 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.598+0000 7f8c74ff9640 1 -- 172.21.15.40:0/3104993988 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f8c70019020 con 0x7f8c78140da0 2023-07-29T13:27:40.600 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.598+0000 7f8c74ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:40.600 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.598+0000 7f8c74ff9640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:40.601 INFO:teuthology.misc.health.smithi040.stdout: 2023-07-29T13:27:40.601 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":11,"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","created":"2023-07-29T13:27:17.185104+0000","modified":"2023-07-29T13:27:28.649513+0000","last_up_change":"0.000000","last_in_change":"2023-07-29T13:27:22.831496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6069a944-26b4-4b00-94e2-f91c485e8957","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab18edd4-e1cb-4815-b5a8-af2b4b2e790b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4575bc2c-4d43-46d3-95ba-1dc2c94278e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e7224640-7d33-4bb7-81a5-99fc26da7763","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23cd76f6-788b-48cf-9438-e931659d3a1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0ddb1e3f-0f65-40d6-ab50-5dc521774445","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4b9d2830-9f2c-438c-9c0b-b6692e21f921","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6593807d-34a1-431b-aa23-24e8340d887b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-29T13:27:40.603 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.600+0000 7f8c7ea41640 10 librados: watch_flush enter 2023-07-29T13:27:40.603 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.600+0000 7f8c7ea41640 10 librados: watch_flush exit 2023-07-29T13:27:40.603 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 20 client.4154.objecter shutdown clearing up homeless session... 2023-07-29T13:27:40.603 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 10 client.4154.objecter successfully canceled tick 2023-07-29T13:27:40.603 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 -- 172.21.15.40:0/3104993988 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f8c6406bed0 msgr2=0x7f8c6406e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:40.603 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f8c6406bed0 0x7f8c6406e380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800aac0 tx=0x7f8c68008040 comp rx=0 tx=0).stop 2023-07-29T13:27:40.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 10 monclient: shutdown 2023-07-29T13:27:40.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:40.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 -- 172.21.15.40:0/3104993988 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c78140da0 msgr2=0x7f8c781d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:40.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c78140da0 0x7f8c781d4a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000b990 tx=0x7f8c7000be50 comp rx=0 tx=0).stop 2023-07-29T13:27:40.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 -- 172.21.15.40:0/3104993988 shutdown_connections 2023-07-29T13:27:40.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f8c6406bed0 0x7f8c6406e380 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:40.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7813dcb0 0x7f8c7806f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:40.605 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8c78140da0 0x7f8c781d4a30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:40.605 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 --2- 172.21.15.40:0/3104993988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780b1a70 0x7f8c780b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:40.605 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 -- 172.21.15.40:0/3104993988 >> 172.21.15.40:0/3104993988 conn(0x7f8c78071680 msgr2=0x7f8c7813b580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:40.605 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 -- 172.21.15.40:0/3104993988 shutdown_connections 2023-07-29T13:27:40.605 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 -- 172.21.15.40:0/3104993988 wait complete. 2023-07-29T13:27:40.605 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:40.601+0000 7f8c7ea41640 1 librados: shutdown 2023-07-29T13:27:40.616 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-29T13:27:41.089 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:17.764 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:41.771 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:18.426 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:41.812 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:18.489 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:42.160 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:18.844 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:42.248 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:18.931 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:42.326 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:19.011 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:42.338 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:19.017 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:43.680 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:20.341 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:46.509 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:23.184 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:46.514 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:23.188 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:46.797 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:23.474 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:46.802 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:23.478 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:47.094 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:23.748 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:47.098 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:23.752 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:47.180 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:23.859 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:47.184 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:23.864 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:47.204 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:23.887 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:47.208 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:23.891 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:47.347 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:24.033 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:47.348 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:24.032 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:47.353 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:24.039 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:47.354 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:24.039 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:47.422 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-29T13:27:47.555 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.554+0000 7f5dd0c48640 10 monclient: get_monmap_and_config 2023-07-29T13:27:47.556 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.554+0000 7f5dd0c48640 10 monclient: build_initial_monmap 2023-07-29T13:27:47.556 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.554+0000 7f5dd0c48640 10 monclient: monmap: 2023-07-29T13:27:47.556 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:27:47.557 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:47.557 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:27:47.555088+0000 2023-07-29T13:27:47.557 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:27:47.555088+0000 2023-07-29T13:27:47.557 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:47.558 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:47.558 INFO:teuthology.misc.health.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:47.558 INFO:teuthology.misc.health.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:47.559 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:47.559 INFO:teuthology.misc.health.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:47.559 INFO:teuthology.misc.health.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:47.559 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:47.563 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.555+0000 7f5dd0c48640 1 Processor -- start 2023-07-29T13:27:47.563 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.555+0000 7f5dd0c48640 1 -- start start 2023-07-29T13:27:47.563 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.555+0000 7f5dd0c48640 10 monclient: init 2023-07-29T13:27:47.564 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:47.564 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2023-07-29T13:27:47.564 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5dcc0b1a70 0x7f5dcc0b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:47.565 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 10 monclient(hunting): picked mon.noname-c con 0x7f5dcc0b1a70 addr v2:172.21.15.45:3300/0 2023-07-29T13:27:47.565 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc13dcb0 0x7f5dcc13e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:47.565 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 10 monclient(hunting): picked mon.noname-b con 0x7f5dcc13dcb0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:47.565 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5dcc140da0 addr v1:172.21.15.40:6789/0 2023-07-29T13:27:47.566 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc13e620 con 0x7f5dcc140da0 2023-07-29T13:27:47.566 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:47.566 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc0a9540 con 0x7f5dcc0b1a70 2023-07-29T13:27:47.566 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:47.567 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0a9740 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.567 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:47.567 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.556+0000 7f5dd0c48640 10 monclient(hunting): authenticate will time out at 1389.688517s 2023-07-29T13:27:47.567 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dc9d9b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc13dcb0 0x7f5dcc13e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:47.568 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dca59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5dcc0b1a70 0x7f5dcc0b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:47.568 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dc9d9b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc13dcb0 0x7f5dcc13e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:53884/0 (socket says 172.21.15.40:53884) 2023-07-29T13:27:47.568 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dc9d9b640 1 -- 172.21.15.40:0/2379709876 learned_addr learned my addr 172.21.15.40:0/2379709876 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:27:47.569 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dc9d9b640 10 monclient(hunting): get_auth_request con 0x7f5dcc13dcb0 auth_method 0 2023-07-29T13:27:47.569 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dc9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:47.569 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dc9d9b640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:47.569 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dc9d9b640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:47.570 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dca59c640 10 monclient(hunting): get_auth_request con 0x7f5dcc0b1a70 auth_method 0 2023-07-29T13:27:47.570 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:47.570 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dca59c640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:47.570 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.557+0000 7f5dca59c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:47.571 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.558+0000 7f5dc9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:47.571 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.558+0000 7f5dc9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:47.571 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.558+0000 7f5dc9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:47.571 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.558+0000 7f5dca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:47.572 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.558+0000 7f5dca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:47.572 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.558+0000 7f5dca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:47.572 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.560+0000 7f5dc9d9b640 10 monclient(hunting): handle_auth_done global_id 4157 payload 274 2023-07-29T13:27:47.573 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.560+0000 7f5dc9d9b640 1 -- 172.21.15.40:0/2379709876 >> v2:172.21.15.45:3300/0 conn(0x7f5dcc0b1a70 msgr2=0x7f5dcc0b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:47.573 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.560+0000 7f5dc9d9b640 1 --2- 172.21.15.40:0/2379709876 >> v2:172.21.15.45:3300/0 conn(0x7f5dcc0b1a70 0x7f5dcc0b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:47.573 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.560+0000 7f5dc9d9b640 1 -- 172.21.15.40:0/2379709876 >> v1:172.21.15.40:6789/0 conn(0x7f5dcc140da0 legacy=0x7f5dcc1411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:47.573 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.560+0000 7f5dc9d9b640 10 monclient: _finish_hunting 0 2023-07-29T13:27:47.573 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.560+0000 7f5dc9d9b640 1 monclient: found mon.noname-b 2023-07-29T13:27:47.573 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.560+0000 7f5dc9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:47.573 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.560+0000 7f5dc9d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:27:47.574 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.560+0000 7f5dc9d9b640 1 -- 172.21.15.40:0/2379709876 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc0a9a20 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.576 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.563+0000 7f5dc9d9b640 1 --2- 172.21.15.40:0/2379709876 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc13dcb0 0x7f5dcc13e0e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009870 tx=0x7f5dc002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bee7269777a637b server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:47.577 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 1 -- 172.21.15.40:0/2379709876 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0031640 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.577 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:47.577 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:27:47.577 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 10 monclient: dump: 2023-07-29T13:27:47.577 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:47.578 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:47.578 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:47.578 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:47.578 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:47.578 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:47.578 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:47.578 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:47.578 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:47.579 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:47.579 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 10 monclient: _finish_auth 0 2023-07-29T13:27:47.579 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 10 monclient: _check_auth_tickets 2023-07-29T13:27:47.579 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:47.579 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 1 -- 172.21.15.40:0/2379709876 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc00317a0 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.579 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:47.579 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 1 -- 172.21.15.40:0/2379709876 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0031b30 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.579 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:47.580 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:27:47.580 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dc959a640 10 monclient: dump: 2023-07-29T13:27:47.580 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:47.580 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:47.580 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:47.580 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:47.580 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:47.581 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:47.581 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:47.581 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:47.581 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:47.581 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:47.581 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dd0c48640 5 monclient: authenticate success, global_id 4157 2023-07-29T13:27:47.581 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dd0c48640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:47.581 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.567+0000 7f5dd0c48640 4 set_mon_vals no callback set 2023-07-29T13:27:47.582 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.568+0000 7f5dd0c48640 10 monclient: shutdown 2023-07-29T13:27:47.582 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.568+0000 7f5dd0c48640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:47.582 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.568+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2379709876 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc13dcb0 msgr2=0x7f5dcc13e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:47.582 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.568+0000 7f5dd0c48640 1 --2- 172.21.15.40:0/2379709876 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc13dcb0 0x7f5dcc13e0e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009870 tx=0x7f5dc002b8f0 comp rx=0 tx=0).stop 2023-07-29T13:27:47.582 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.569+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2379709876 shutdown_connections 2023-07-29T13:27:47.582 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.569+0000 7f5dd0c48640 1 --2- 172.21.15.40:0/2379709876 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc13dcb0 0x7f5dcc13e0e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:47.582 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.569+0000 7f5dd0c48640 1 --2- 172.21.15.40:0/2379709876 >> v2:172.21.15.45:3300/0 conn(0x7f5dcc0b1a70 0x7f5dcc0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:47.583 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.569+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2379709876 >> 172.21.15.40:0/2379709876 conn(0x7f5dcc071680 msgr2=0x7f5dcc0659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:47.583 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.569+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2379709876 shutdown_connections 2023-07-29T13:27:47.583 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.569+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2379709876 wait complete. 2023-07-29T13:27:47.583 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.570+0000 7f5dd0c48640 10 monclient: build_initial_monmap 2023-07-29T13:27:47.583 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.570+0000 7f5dd0c48640 10 monclient: monmap: 2023-07-29T13:27:47.583 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:27:47.583 INFO:teuthology.misc.health.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:47.583 INFO:teuthology.misc.health.smithi040.stderr:last_changed 0.000000 2023-07-29T13:27:47.584 INFO:teuthology.misc.health.smithi040.stderr:created 0.000000 2023-07-29T13:27:47.584 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:47.584 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:47.584 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:47.584 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:47.584 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:47.585 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:47.585 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.570+0000 7f5dd0c48640 1 librados: starting msgr at 2023-07-29T13:27:47.585 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.570+0000 7f5dd0c48640 1 librados: starting objecter 2023-07-29T13:27:47.585 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 1 Processor -- start 2023-07-29T13:27:47.585 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 1 -- start start 2023-07-29T13:27:47.585 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 1 librados: setting wanted keys 2023-07-29T13:27:47.585 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 1 librados: calling monclient init 2023-07-29T13:27:47.585 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 10 monclient: init 2023-07-29T13:27:47.586 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:47.586 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:27:47.586 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc0b1a70 0x7f5dcc1d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:47.586 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 10 monclient(hunting): picked mon.noname-c con 0x7f5dcc0b1a70 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:47.586 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc13dcb0 0x7f5dcc1d63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:47.587 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 10 monclient(hunting): picked mon.noname-b con 0x7f5dcc13dcb0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:47.587 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.571+0000 7f5dd0c48640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dcc140da0 0x7f5dcc0abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:47.587 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dd0c48640 10 monclient(hunting): picked mon.noname-a con 0x7f5dcc140da0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:47.587 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dd0c48640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:47.588 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dd0c48640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0aa820 con 0x7f5dcc140da0 2023-07-29T13:27:47.588 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dd0c48640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:47.588 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dd0c48640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc1d59b0 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.589 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dd0c48640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:47.589 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dd0c48640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc143920 con 0x7f5dcc0b1a70 2023-07-29T13:27:47.589 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dd0c48640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:47.589 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dd0c48640 10 monclient(hunting): authenticate will time out at 1389.704254s 2023-07-29T13:27:47.590 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dca59c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc0b1a70 0x7f5dcc1d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:47.590 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dc9d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc13dcb0 0x7f5dcc1d63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:47.590 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.572+0000 7f5dcad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dcc140da0 0x7f5dcc0abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:47.590 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dca59c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc0b1a70 0x7f5dcc1d5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:53898/0 (socket says 172.21.15.40:53898) 2023-07-29T13:27:47.591 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dca59c640 1 -- 172.21.15.40:0/2735945766 learned_addr learned my addr 172.21.15.40:0/2735945766 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:27:47.591 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dca59c640 10 monclient(hunting): get_auth_request con 0x7f5dcc0b1a70 auth_method 0 2023-07-29T13:27:47.591 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:47.591 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dca59c640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:47.592 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dca59c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:47.592 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dc9d9b640 10 monclient(hunting): get_auth_request con 0x7f5dcc13dcb0 auth_method 0 2023-07-29T13:27:47.592 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dc9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:47.592 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dc9d9b640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:47.593 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dc9d9b640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:47.593 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dcad9d640 10 monclient(hunting): get_auth_request con 0x7f5dcc140da0 auth_method 0 2023-07-29T13:27:47.593 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dcad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:47.593 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dcad9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:47.593 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dcad9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:47.593 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:47.593 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:47.593 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:47.594 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dc9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:47.594 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dc9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:47.594 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.573+0000 7f5dc9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:47.594 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.574+0000 7f5dcad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:47.594 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.574+0000 7f5dcad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:47.594 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.574+0000 7f5dcad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:47.597 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.577+0000 7f5dc9d9b640 10 monclient(hunting): handle_auth_done global_id 4219 payload 824 2023-07-29T13:27:47.598 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.577+0000 7f5dc9d9b640 1 -- 172.21.15.40:0/2735945766 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc0b1a70 msgr2=0x7f5dcc1d5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:47.598 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.577+0000 7f5dc9d9b640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc0b1a70 0x7f5dcc1d5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:47.598 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.577+0000 7f5dc9d9b640 1 -- 172.21.15.40:0/2735945766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dcc140da0 msgr2=0x7f5dcc0abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:47.598 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.577+0000 7f5dc9d9b640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dcc140da0 0x7f5dcc0abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:47.599 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.578+0000 7f5dc9d9b640 10 monclient: _finish_hunting 0 2023-07-29T13:27:47.599 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.578+0000 7f5dc9d9b640 1 monclient: found mon.noname-b 2023-07-29T13:27:47.599 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.578+0000 7f5dc9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:47.599 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.578+0000 7f5dc9d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:27:47.600 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.578+0000 7f5dc9d9b640 1 -- 172.21.15.40:0/2735945766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc00094d0 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.600 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.581+0000 7f5dc9d9b640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc13dcb0 0x7f5dcc1d63d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0033000 tx=0x7f5dc00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:47.600 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.581+0000 7f5dbaffd640 10 client.?.objecter ms_handle_connect 0x7f5dcc13dcb0 2023-07-29T13:27:47.601 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.581+0000 7f5dbaffd640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:47.601 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 1 -- 172.21.15.40:0/2735945766 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc003ac60 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.601 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:47.601 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:27:47.601 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 10 monclient: dump: 2023-07-29T13:27:47.602 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:47.602 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:47.602 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:47.602 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:47.602 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:47.602 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:47.603 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:47.603 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:47.603 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:47.603 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:47.603 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 10 monclient: _finish_auth 0 2023-07-29T13:27:47.603 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 10 monclient: _check_auth_tickets 2023-07-29T13:27:47.603 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:47.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 1 -- 172.21.15.40:0/2735945766 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc003ade0 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:47.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 1 -- 172.21.15.40:0/2735945766 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0044630 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:47.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:47.604 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dbaffd640 10 monclient: dump: 2023-07-29T13:27:47.605 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:47.605 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:47.605 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:47.605 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:47.605 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:47.606 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:47.606 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:47.606 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:47.606 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:47.606 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:47.606 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dd0c48640 5 monclient: authenticate success, global_id 4219 2023-07-29T13:27:47.607 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dd0c48640 10 monclient: _renew_subs 2023-07-29T13:27:47.607 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dd0c48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:47.607 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dc8d99640 4 set_mon_vals no callback set 2023-07-29T13:27:47.607 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2735945766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc0ab2d0 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.607 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dd0c48640 10 client.4219.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:47.607 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dd0c48640 10 monclient: _renew_subs 2023-07-29T13:27:47.607 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dd0c48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:47.607 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2735945766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc0ac520 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.608 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.586+0000 7f5dd0c48640 1 librados: init done 2023-07-29T13:27:47.608 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.587+0000 7f5dd0c48640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:47.608 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.587+0000 7f5dd0c48640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:47.608 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.587+0000 7f5dd0c48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:47.608 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.587+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2735945766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc1d5b70 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.958 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.957+0000 7f5dbaffd640 1 -- 172.21.15.40:0/2735945766 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 6) v1 ==== 88153+0+0 (secure 0 0 0) 0x7f5dc002f050 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.958 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.958+0000 7f5dbaffd640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f5dac06bed0 0x7f5dac06e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:47.959 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.958+0000 7f5dca59c640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f5dac06bed0 0x7f5dac06e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:47.959 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.958+0000 7f5dca59c640 10 monclient: get_auth_request con 0x7f5dac06bed0 auth_method 0 2023-07-29T13:27:47.959 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.958+0000 7f5dca59c640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f5dac06bed0 0x7f5dac06e380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009c00 tx=0x7f5dbc009190 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:47.959 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.958+0000 7f5dbaffd640 10 client.4219.objecter ms_handle_connect 0x7f5dac06bed0 2023-07-29T13:27:47.978 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.977+0000 7f5dbaffd640 1 -- 172.21.15.40:0/2735945766 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5dc007dc80 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.979 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.977+0000 7f5dbaffd640 10 client.4219.objecter ms_dispatch 0x7f5dcc000b90 osd_map(11..11 src has 1..11) v4 2023-07-29T13:27:47.979 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.977+0000 7f5dbaffd640 3 client.4219.objecter handle_osd_map got epochs [11,11] > 0 2023-07-29T13:27:47.979 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.977+0000 7f5dbaffd640 3 client.4219.objecter handle_osd_map decoding full epoch 11 2023-07-29T13:27:47.979 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.977+0000 7f5dbaffd640 20 client.4219.objecter dump_active .. 0 homeless 2023-07-29T13:27:47.980 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.977+0000 7f5dbaffd640 1 -- 172.21.15.40:0/2735945766 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f5dc00760b0 con 0x7f5dcc13dcb0 2023-07-29T13:27:47.980 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.977+0000 7f5dbaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:47.980 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:47.977+0000 7f5dbaffd640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:48.195 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.194+0000 7f5dd0c48640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:48.196 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.194+0000 7f5dd0c48640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:48.196 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.194+0000 7f5dd0c48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:27:48.196 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.194+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2735945766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dcc143ae0 con 0x7f5dcc13dcb0 2023-07-29T13:27:48.224 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.223+0000 7f5dbaffd640 1 -- 172.21.15.40:0/2735945766 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f5dc0035090 con 0x7f5dcc13dcb0 2023-07-29T13:27:48.225 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.223+0000 7f5dbaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:48.225 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.223+0000 7f5dbaffd640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:48.225 INFO:teuthology.misc.health.smithi040.stdout: 2023-07-29T13:27:48.225 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":11,"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","created":"2023-07-29T13:27:17.185104+0000","modified":"2023-07-29T13:27:28.649513+0000","last_up_change":"0.000000","last_in_change":"2023-07-29T13:27:22.831496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6069a944-26b4-4b00-94e2-f91c485e8957","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab18edd4-e1cb-4815-b5a8-af2b4b2e790b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4575bc2c-4d43-46d3-95ba-1dc2c94278e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e7224640-7d33-4bb7-81a5-99fc26da7763","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23cd76f6-788b-48cf-9438-e931659d3a1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0ddb1e3f-0f65-40d6-ab50-5dc521774445","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4b9d2830-9f2c-438c-9c0b-b6692e21f921","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6593807d-34a1-431b-aa23-24e8340d887b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-29T13:27:48.227 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.226+0000 7f5dd0c48640 10 librados: watch_flush enter 2023-07-29T13:27:48.227 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.226+0000 7f5dd0c48640 10 librados: watch_flush exit 2023-07-29T13:27:48.227 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.226+0000 7f5dd0c48640 20 client.4219.objecter shutdown clearing up homeless session... 2023-07-29T13:27:48.227 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.226+0000 7f5dd0c48640 10 client.4219.objecter successfully canceled tick 2023-07-29T13:27:48.229 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2735945766 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f5dac06bed0 msgr2=0x7f5dac06e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:48.229 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f5dac06bed0 0x7f5dac06e380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009c00 tx=0x7f5dbc009190 comp rx=0 tx=0).stop 2023-07-29T13:27:48.229 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 10 monclient: shutdown 2023-07-29T13:27:48.229 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:48.229 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2735945766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc13dcb0 msgr2=0x7f5dcc1d63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:48.229 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc13dcb0 0x7f5dcc1d63d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0033000 tx=0x7f5dc00046c0 comp rx=0 tx=0).stop 2023-07-29T13:27:48.230 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2735945766 shutdown_connections 2023-07-29T13:27:48.230 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f5dac06bed0 0x7f5dac06e380 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:48.230 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc13dcb0 0x7f5dcc1d63d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:48.230 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dcc140da0 0x7f5dcc0abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:48.230 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 --2- 172.21.15.40:0/2735945766 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5dcc0b1a70 0x7f5dcc1d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:48.230 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2735945766 >> 172.21.15.40:0/2735945766 conn(0x7f5dcc071680 msgr2=0x7f5dcc0659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:48.230 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2735945766 shutdown_connections 2023-07-29T13:27:48.231 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 -- 172.21.15.40:0/2735945766 wait complete. 2023-07-29T13:27:48.231 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:48.228+0000 7f5dd0c48640 1 librados: shutdown 2023-07-29T13:27:48.242 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-29T13:27:48.819 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:25.480 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:48.823 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:25.485 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:50.478 INFO:tasks.ceph.osd.7.smithi045.stderr:==00:00:00:27.153 85182== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:51.143 INFO:tasks.ceph.osd.5.smithi045.stderr:==00:00:00:27.819 85181== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:51.577 INFO:tasks.ceph.osd.6.smithi045.stderr:==00:00:00:28.231 85184== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:51.886 INFO:tasks.ceph.osd.1.smithi040.stderr:==00:00:00:28.565 85573== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:51.900 INFO:tasks.ceph.osd.2.smithi040.stderr:==00:00:00:28.583 85572== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:52.105 INFO:tasks.ceph.osd.0.smithi040.stderr:==00:00:00:28.789 85570== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:52.255 INFO:tasks.ceph.osd.3.smithi040.stderr:==00:00:00:28.941 85571== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:52.794 INFO:tasks.ceph.osd.4.smithi045.stderr:==00:00:00:29.456 85183== Warning: unimplemented fcntl command: 1038 2023-07-29T13:27:55.047 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-29T13:27:55.157 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.156+0000 7fd9906d5640 10 monclient: get_monmap_and_config 2023-07-29T13:27:55.158 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.156+0000 7fd9906d5640 10 monclient: build_initial_monmap 2023-07-29T13:27:55.158 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.156+0000 7fd9906d5640 10 monclient: monmap: 2023-07-29T13:27:55.158 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:27:55.159 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:55.159 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:27:55.156413+0000 2023-07-29T13:27:55.159 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:27:55.156413+0000 2023-07-29T13:27:55.160 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:55.160 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:55.160 INFO:teuthology.misc.health.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:27:55.160 INFO:teuthology.misc.health.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:27:55.160 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:27:55.160 INFO:teuthology.misc.health.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:27:55.161 INFO:teuthology.misc.health.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:27:55.161 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:55.161 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.156+0000 7fd9906d5640 1 Processor -- start 2023-07-29T13:27:55.161 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.156+0000 7fd9906d5640 1 -- start start 2023-07-29T13:27:55.161 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.156+0000 7fd9906d5640 10 monclient: init 2023-07-29T13:27:55.163 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.157+0000 7fd9906d5640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:55.163 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.157+0000 7fd9906d5640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2023-07-29T13:27:55.163 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.157+0000 7fd9906d5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd9880b1a70 0x7fd9880b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:55.164 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.157+0000 7fd9906d5640 10 monclient(hunting): picked mon.noname-b con 0x7fd9880b1a70 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:55.164 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.157+0000 7fd9906d5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd98813dcb0 addr v1:172.21.15.45:6789/0 2023-07-29T13:27:55.164 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.157+0000 7fd9906d5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd988140da0 0x7fd9881411b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:55.164 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.157+0000 7fd9906d5640 10 monclient(hunting): picked mon.noname-c con 0x7fd988140da0 addr v2:172.21.15.45:3300/0 2023-07-29T13:27:55.165 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.157+0000 7fd9906d5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9880a9740 con 0x7fd98813dcb0 2023-07-29T13:27:55.165 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.158+0000 7fd9906d5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:55.165 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.158+0000 7fd9906d5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9880a9540 con 0x7fd988140da0 2023-07-29T13:27:55.165 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.158+0000 7fd9906d5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:55.166 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.158+0000 7fd9906d5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880a9a20 con 0x7fd9880b1a70 2023-07-29T13:27:55.166 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.158+0000 7fd9906d5640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:55.166 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.158+0000 7fd9906d5640 10 monclient(hunting): authenticate will time out at 1397.290172s 2023-07-29T13:27:55.167 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.158+0000 7fd98ec72640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd988140da0 0x7fd9881411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:55.167 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.158+0000 7fd98e471640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd9880b1a70 0x7fd9880b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:55.167 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98e471640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd9880b1a70 0x7fd9880b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:38906/0 (socket says 172.21.15.40:38906) 2023-07-29T13:27:55.167 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98e471640 1 -- 172.21.15.40:0/1539548266 learned_addr learned my addr 172.21.15.40:0/1539548266 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:27:55.168 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98e471640 10 monclient(hunting): get_auth_request con 0x7fd9880b1a70 auth_method 0 2023-07-29T13:27:55.168 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98e471640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:55.168 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98e471640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:55.168 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98e471640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:55.169 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98ec72640 10 monclient(hunting): get_auth_request con 0x7fd988140da0 auth_method 0 2023-07-29T13:27:55.169 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98ec72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:55.169 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98ec72640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:55.170 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98ec72640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:55.170 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98e471640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:55.170 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98e471640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:55.170 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.159+0000 7fd98e471640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:55.171 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.160+0000 7fd98ec72640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:55.171 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.160+0000 7fd98ec72640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:55.171 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.160+0000 7fd98ec72640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:55.172 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.163+0000 7fd98ec72640 10 monclient(hunting): handle_auth_done global_id 4222 payload 274 2023-07-29T13:27:55.173 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.163+0000 7fd98ec72640 1 -- 172.21.15.40:0/1539548266 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd9880b1a70 msgr2=0x7fd9880b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:27:55.173 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.163+0000 7fd98ec72640 1 --2- 172.21.15.40:0/1539548266 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd9880b1a70 0x7fd9880b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:55.173 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.163+0000 7fd98ec72640 1 -- 172.21.15.40:0/1539548266 >> v1:172.21.15.45:6789/0 conn(0x7fd98813dcb0 legacy=0x7fd98813e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:55.173 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.163+0000 7fd98ec72640 10 monclient: _finish_hunting 0 2023-07-29T13:27:55.174 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.163+0000 7fd98ec72640 1 monclient: found mon.noname-c 2023-07-29T13:27:55.174 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.163+0000 7fd98ec72640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:55.174 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.163+0000 7fd98ec72640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:27:55.174 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.163+0000 7fd98ec72640 1 -- 172.21.15.40:0/1539548266 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9880a9d00 con 0x7fd988140da0 2023-07-29T13:27:55.175 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.168+0000 7fd98ec72640 1 --2- 172.21.15.40:0/1539548266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd988140da0 0x7fd9881411b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd980009d90 tx=0x7fd98002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c197cb66a2f99632 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:55.179 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 1 -- 172.21.15.40:0/1539548266 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98002cbc0 con 0x7fd988140da0 2023-07-29T13:27:55.179 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:55.179 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:27:55.179 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 10 monclient: dump: 2023-07-29T13:27:55.180 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:55.180 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:55.180 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:55.180 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:55.181 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:55.181 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:55.181 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:55.181 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:55.182 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:55.182 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:55.182 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 10 monclient: _finish_auth 0 2023-07-29T13:27:55.182 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 10 monclient: _check_auth_tickets 2023-07-29T13:27:55.183 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:55.183 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 1 -- 172.21.15.40:0/1539548266 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98002cd20 con 0x7fd988140da0 2023-07-29T13:27:55.183 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd9906d5640 5 monclient: authenticate success, global_id 4222 2023-07-29T13:27:55.183 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd9906d5640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:27:55.184 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:55.184 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd9906d5640 10 monclient: get_monmap_and_config success 2023-07-29T13:27:55.184 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 1 -- 172.21.15.40:0/1539548266 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980034660 con 0x7fd988140da0 2023-07-29T13:27:55.184 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd9906d5640 4 set_mon_vals no callback set 2023-07-29T13:27:55.184 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:55.185 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:27:55.185 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.173+0000 7fd98d46f640 10 monclient: dump: 2023-07-29T13:27:55.185 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:55.185 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:55.186 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:55.186 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:55.186 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:55.186 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:55.186 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:55.187 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:55.187 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:55.187 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:55.187 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.174+0000 7fd9906d5640 10 monclient: shutdown 2023-07-29T13:27:55.187 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.174+0000 7fd9906d5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:55.188 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.174+0000 7fd9906d5640 1 -- 172.21.15.40:0/1539548266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd988140da0 msgr2=0x7fd9881411b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:55.188 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.174+0000 7fd9906d5640 1 --2- 172.21.15.40:0/1539548266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd988140da0 0x7fd9881411b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd980009d90 tx=0x7fd98002be10 comp rx=0 tx=0).stop 2023-07-29T13:27:55.188 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.174+0000 7fd9906d5640 1 -- 172.21.15.40:0/1539548266 shutdown_connections 2023-07-29T13:27:55.188 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.174+0000 7fd9906d5640 1 --2- 172.21.15.40:0/1539548266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd988140da0 0x7fd9881411b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:55.189 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.174+0000 7fd9906d5640 1 --2- 172.21.15.40:0/1539548266 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd9880b1a70 0x7fd9880b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:55.189 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.174+0000 7fd9906d5640 1 -- 172.21.15.40:0/1539548266 >> 172.21.15.40:0/1539548266 conn(0x7fd988071680 msgr2=0x7fd9880659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:55.189 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.174+0000 7fd9906d5640 1 -- 172.21.15.40:0/1539548266 shutdown_connections 2023-07-29T13:27:55.189 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.174+0000 7fd9906d5640 1 -- 172.21.15.40:0/1539548266 wait complete. 2023-07-29T13:27:55.190 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.175+0000 7fd9906d5640 10 monclient: build_initial_monmap 2023-07-29T13:27:55.190 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.175+0000 7fd9906d5640 10 monclient: monmap: 2023-07-29T13:27:55.190 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:27:55.190 INFO:teuthology.misc.health.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:27:55.190 INFO:teuthology.misc.health.smithi040.stderr:last_changed 0.000000 2023-07-29T13:27:55.191 INFO:teuthology.misc.health.smithi040.stderr:created 0.000000 2023-07-29T13:27:55.191 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:27:55.191 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:55.191 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:27:55.191 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:27:55.192 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:27:55.192 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:55.192 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.175+0000 7fd9906d5640 1 librados: starting msgr at 2023-07-29T13:27:55.192 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.175+0000 7fd9906d5640 1 librados: starting objecter 2023-07-29T13:27:55.193 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.176+0000 7fd9906d5640 1 Processor -- start 2023-07-29T13:27:55.193 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.176+0000 7fd9906d5640 1 -- start start 2023-07-29T13:27:55.193 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.176+0000 7fd9906d5640 1 librados: setting wanted keys 2023-07-29T13:27:55.193 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.176+0000 7fd9906d5640 1 librados: calling monclient init 2023-07-29T13:27:55.194 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.176+0000 7fd9906d5640 10 monclient: init 2023-07-29T13:27:55.194 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.176+0000 7fd9906d5640 10 monclient: _reopen_session rank -1 2023-07-29T13:27:55.194 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.176+0000 7fd9906d5640 10 monclient: _add_conns ranks=[1,0,2] 2023-07-29T13:27:55.194 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9880b1a70 0x7fd988143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:55.194 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 10 monclient(hunting): picked mon.noname-b con 0x7fd9880b1a70 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:27:55.195 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd98813dcb0 0x7fd9880aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:55.195 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 10 monclient(hunting): picked mon.noname-a con 0x7fd98813dcb0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:27:55.195 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd988140da0 0x7fd9881d8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:55.195 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 10 monclient(hunting): picked mon.noname-c con 0x7fd988140da0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:27:55.195 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:55.196 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880ac360 con 0x7fd98813dcb0 2023-07-29T13:27:55.196 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:55.196 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880abfd0 con 0x7fd9880b1a70 2023-07-29T13:27:55.196 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:27:55.196 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880ac7f0 con 0x7fd988140da0 2023-07-29T13:27:55.196 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 10 monclient(hunting): _renew_subs 2023-07-29T13:27:55.196 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd9906d5640 10 monclient(hunting): authenticate will time out at 1397.309180s 2023-07-29T13:27:55.197 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd98ec72640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd988140da0 0x7fd9881d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:55.197 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd98dc70640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd98813dcb0 0x7fd9880aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:55.197 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.177+0000 7fd98e471640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9880b1a70 0x7fd988143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:55.197 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98ec72640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd988140da0 0x7fd9881d8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:38908/0 (socket says 172.21.15.40:38908) 2023-07-29T13:27:55.197 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98ec72640 1 -- 172.21.15.40:0/641923141 learned_addr learned my addr 172.21.15.40:0/641923141 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:27:55.197 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98ec72640 10 monclient(hunting): get_auth_request con 0x7fd988140da0 auth_method 0 2023-07-29T13:27:55.197 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98ec72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:55.198 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98ec72640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:55.198 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98ec72640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:55.199 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98e471640 10 monclient(hunting): get_auth_request con 0x7fd9880b1a70 auth_method 0 2023-07-29T13:27:55.200 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98e471640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:55.200 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98e471640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:55.200 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98e471640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:55.200 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98dc70640 10 monclient(hunting): get_auth_request con 0x7fd98813dcb0 auth_method 0 2023-07-29T13:27:55.200 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98dc70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:27:55.200 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98dc70640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:27:55.201 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98dc70640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:27:55.201 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98ec72640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:55.201 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98ec72640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:55.201 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.178+0000 7fd98ec72640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:55.201 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.179+0000 7fd98e471640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:55.201 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.179+0000 7fd98e471640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:55.201 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.179+0000 7fd98e471640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:55.201 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.179+0000 7fd98dc70640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:27:55.202 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.179+0000 7fd98dc70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:27:55.202 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.179+0000 7fd98dc70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:27:55.202 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.182+0000 7fd98ec72640 10 monclient(hunting): handle_auth_done global_id 4166 payload 824 2023-07-29T13:27:55.202 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.182+0000 7fd98ec72640 1 -- 172.21.15.40:0/641923141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9880b1a70 msgr2=0x7fd988143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:55.202 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.182+0000 7fd98ec72640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9880b1a70 0x7fd988143c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:55.202 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.182+0000 7fd98ec72640 1 -- 172.21.15.40:0/641923141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd98813dcb0 msgr2=0x7fd9880aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:55.202 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.182+0000 7fd98ec72640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd98813dcb0 0x7fd9880aba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:55.203 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.182+0000 7fd98ec72640 10 monclient: _finish_hunting 0 2023-07-29T13:27:55.203 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.182+0000 7fd98ec72640 1 monclient: found mon.noname-c 2023-07-29T13:27:55.203 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.183+0000 7fd98ec72640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:27:55.203 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.183+0000 7fd98ec72640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:27:55.203 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.183+0000 7fd98ec72640 1 -- 172.21.15.40:0/641923141 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd980009a70 con 0x7fd988140da0 2023-07-29T13:27:55.203 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.186+0000 7fd98ec72640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd988140da0 0x7fd9881d8f00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd980009d90 tx=0x7fd980004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:55.203 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.186+0000 7fd97effd640 10 client.?.objecter ms_handle_connect 0x7fd988140da0 2023-07-29T13:27:55.203 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.186+0000 7fd97effd640 10 client.?.objecter resend_mon_ops 2023-07-29T13:27:55.204 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd97effd640 1 -- 172.21.15.40:0/641923141 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980040650 con 0x7fd988140da0 2023-07-29T13:27:55.204 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd97effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:55.204 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd97effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:27:55.204 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd97effd640 10 monclient: dump: 2023-07-29T13:27:55.204 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:55.204 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:55.204 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:55.205 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:55.205 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:55.205 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:55.205 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:55.205 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:55.205 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:55.206 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:55.206 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd97effd640 10 monclient: _finish_auth 0 2023-07-29T13:27:55.206 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd97effd640 10 monclient: _check_auth_tickets 2023-07-29T13:27:55.206 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd97effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:27:55.207 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd97effd640 1 -- 172.21.15.40:0/641923141 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98002ce90 con 0x7fd988140da0 2023-07-29T13:27:55.207 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd97effd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:27:55.207 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd98cc6e640 4 set_mon_vals no callback set 2023-07-29T13:27:55.207 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd9906d5640 5 monclient: authenticate success, global_id 4166 2023-07-29T13:27:55.208 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd97effd640 1 -- 172.21.15.40:0/641923141 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800409f0 con 0x7fd988140da0 2023-07-29T13:27:55.208 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd9906d5640 10 monclient: _renew_subs 2023-07-29T13:27:55.208 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd9906d5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:55.208 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.192+0000 7fd9906d5640 1 -- 172.21.15.40:0/641923141 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd988144900 con 0x7fd988140da0 2023-07-29T13:27:55.208 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.193+0000 7fd97effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:27:55.208 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.193+0000 7fd97effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:27:55.208 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.193+0000 7fd9906d5640 10 client.4166.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:27:55.209 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.193+0000 7fd97effd640 10 monclient: dump: 2023-07-29T13:27:55.209 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:27:55.209 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:27:55.209 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:55.209 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:27:55.209 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:27:55.209 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:27:55.209 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:27:55.210 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:27:55.210 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:27:55.210 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:27:55.210 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.193+0000 7fd9906d5640 10 monclient: _renew_subs 2023-07-29T13:27:55.210 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.193+0000 7fd9906d5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:55.210 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.193+0000 7fd9906d5640 1 -- 172.21.15.40:0/641923141 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9881d9620 con 0x7fd988140da0 2023-07-29T13:27:55.210 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.193+0000 7fd9906d5640 1 librados: init done 2023-07-29T13:27:55.210 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.194+0000 7fd9906d5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:55.211 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.194+0000 7fd9906d5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:55.211 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.194+0000 7fd9906d5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:55.211 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.194+0000 7fd9906d5640 1 -- 172.21.15.40:0/641923141 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9880ad8f0 con 0x7fd988140da0 2023-07-29T13:27:55.227 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.227+0000 7fd97effd640 1 -- 172.21.15.40:0/641923141 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 6) v1 ==== 88153+0+0 (secure 0 0 0) 0x7fd980040b50 con 0x7fd988140da0 2023-07-29T13:27:55.228 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.227+0000 7fd97effd640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd96c06bf00 0x7fd96c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:27:55.228 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.227+0000 7fd98e471640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd96c06bf00 0x7fd96c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:27:55.229 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.227+0000 7fd98e471640 10 monclient: get_auth_request con 0x7fd96c06bf00 auth_method 0 2023-07-29T13:27:55.229 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.228+0000 7fd97effd640 10 client.4166.objecter ms_handle_connect 0x7fd96c06bf00 2023-07-29T13:27:55.229 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.228+0000 7fd98e471640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd96c06bf00 0x7fd96c06e3b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd984009b30 tx=0x7fd984009240 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:27:55.271 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.270+0000 7fd97effd640 1 -- 172.21.15.40:0/641923141 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd98007ea10 con 0x7fd988140da0 2023-07-29T13:27:55.271 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.270+0000 7fd97effd640 10 client.4166.objecter ms_dispatch 0x7fd988000b90 osd_map(11..11 src has 1..11) v4 2023-07-29T13:27:55.271 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.270+0000 7fd97effd640 3 client.4166.objecter handle_osd_map got epochs [11,11] > 0 2023-07-29T13:27:55.272 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.270+0000 7fd97effd640 3 client.4166.objecter handle_osd_map decoding full epoch 11 2023-07-29T13:27:55.272 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.270+0000 7fd97effd640 20 client.4166.objecter dump_active .. 0 homeless 2023-07-29T13:27:55.499 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.498+0000 7fd97effd640 1 -- 172.21.15.40:0/641923141 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7fd980076e40 con 0x7fd988140da0 2023-07-29T13:27:55.499 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.498+0000 7fd97effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:27:55.499 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.498+0000 7fd97effd640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:27:55.681 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.680+0000 7fd9906d5640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:55.681 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.680+0000 7fd9906d5640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:55.682 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.680+0000 7fd9906d5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:27:55.682 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.680+0000 7fd9906d5640 1 -- 172.21.15.40:0/641923141 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9880ac140 con 0x7fd988140da0 2023-07-29T13:27:55.698 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.697+0000 7fd97effd640 1 -- 172.21.15.40:0/641923141 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd980009450 con 0x7fd988140da0 2023-07-29T13:27:55.698 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.697+0000 7fd97effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:27:55.699 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.697+0000 7fd97effd640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:27:55.699 INFO:teuthology.misc.health.smithi040.stdout: 2023-07-29T13:27:55.699 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":11,"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","created":"2023-07-29T13:27:17.185104+0000","modified":"2023-07-29T13:27:28.649513+0000","last_up_change":"0.000000","last_in_change":"2023-07-29T13:27:22.831496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6069a944-26b4-4b00-94e2-f91c485e8957","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab18edd4-e1cb-4815-b5a8-af2b4b2e790b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4575bc2c-4d43-46d3-95ba-1dc2c94278e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e7224640-7d33-4bb7-81a5-99fc26da7763","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23cd76f6-788b-48cf-9438-e931659d3a1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0ddb1e3f-0f65-40d6-ab50-5dc521774445","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4b9d2830-9f2c-438c-9c0b-b6692e21f921","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6593807d-34a1-431b-aa23-24e8340d887b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-29T13:27:55.701 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 10 librados: watch_flush enter 2023-07-29T13:27:55.701 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 10 librados: watch_flush exit 2023-07-29T13:27:55.701 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 20 client.4166.objecter shutdown clearing up homeless session... 2023-07-29T13:27:55.701 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 10 client.4166.objecter successfully canceled tick 2023-07-29T13:27:55.702 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 -- 172.21.15.40:0/641923141 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd96c06bf00 msgr2=0x7fd96c06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:55.702 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd96c06bf00 0x7fd96c06e3b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd984009b30 tx=0x7fd984009240 comp rx=0 tx=0).stop 2023-07-29T13:27:55.702 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 10 monclient: shutdown 2023-07-29T13:27:55.702 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:27:55.702 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 -- 172.21.15.40:0/641923141 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd988140da0 msgr2=0x7fd9881d8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:27:55.702 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd988140da0 0x7fd9881d8f00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd980009d90 tx=0x7fd980004a40 comp rx=0 tx=0).stop 2023-07-29T13:27:55.702 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 -- 172.21.15.40:0/641923141 shutdown_connections 2023-07-29T13:27:55.703 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd96c06bf00 0x7fd96c06e3b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:55.703 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd988140da0 0x7fd9881d8f00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:55.703 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd98813dcb0 0x7fd9880aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:55.703 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 --2- 172.21.15.40:0/641923141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9880b1a70 0x7fd988143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:27:55.703 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 -- 172.21.15.40:0/641923141 >> 172.21.15.40:0/641923141 conn(0x7fd988071680 msgr2=0x7fd98813e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:27:55.703 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 -- 172.21.15.40:0/641923141 shutdown_connections 2023-07-29T13:27:55.703 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 -- 172.21.15.40:0/641923141 wait complete. 2023-07-29T13:27:55.703 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:27:55.700+0000 7fd9906d5640 1 librados: shutdown 2023-07-29T13:27:55.714 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-29T13:27:58.518 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:27:58.513+0000 5dc1300 -1 osd.7 0 log_to_monitors true 2023-07-29T13:27:59.441 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:27:59.429+0000 5dc1300 -1 osd.5 0 log_to_monitors true 2023-07-29T13:27:59.720 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:27:59.715+0000 5dc1300 -1 osd.6 0 log_to_monitors true 2023-07-29T13:28:00.483 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:28:00.480+0000 5dc1300 -1 osd.1 0 log_to_monitors true 2023-07-29T13:28:00.506 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:28:00.501+0000 5dc1300 -1 osd.2 0 log_to_monitors true 2023-07-29T13:28:00.648 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:28:00.644+0000 5dc1300 -1 osd.0 0 log_to_monitors true 2023-07-29T13:28:01.045 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:28:01.035+0000 5dc1300 -1 osd.4 0 log_to_monitors true 2023-07-29T13:28:01.178 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:28:01.174+0000 5dc1300 -1 osd.3 0 log_to_monitors true 2023-07-29T13:28:01.764 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:28:01.759+0000 c495640 -1 osd.7 0 waiting for initial osdmap 2023-07-29T13:28:02.242 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:28:02.221+0000 14264640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:28:02.521 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-29T13:28:02.615 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.613+0000 7f118be42640 10 monclient: get_monmap_and_config 2023-07-29T13:28:02.615 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.613+0000 7f118be42640 10 monclient: build_initial_monmap 2023-07-29T13:28:02.616 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.613+0000 7f118be42640 10 monclient: monmap: 2023-07-29T13:28:02.616 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:28:02.616 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:02.616 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:28:02.614911+0000 2023-07-29T13:28:02.616 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:28:02.614911+0000 2023-07-29T13:28:02.616 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:02.616 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:02.617 INFO:teuthology.misc.health.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:02.617 INFO:teuthology.misc.health.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:02.617 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:02.617 INFO:teuthology.misc.health.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:02.617 INFO:teuthology.misc.health.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:02.617 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:02.619 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.614+0000 7f118be42640 1 Processor -- start 2023-07-29T13:28:02.619 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.614+0000 7f118be42640 1 -- start start 2023-07-29T13:28:02.619 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.614+0000 7f118be42640 10 monclient: init 2023-07-29T13:28:02.619 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:02.619 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2023-07-29T13:28:02.619 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f11840b1a70 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:02.620 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f118413e3c0 0x7f11840b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:02.620 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 10 monclient(hunting): picked mon.noname-c con 0x7f118413e3c0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:02.620 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1184140da0 0x7f1184141200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:02.620 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 10 monclient(hunting): picked mon.noname-b con 0x7f1184140da0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:02.620 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11840a9740 con 0x7f11840b1a70 2023-07-29T13:28:02.620 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:02.620 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11840a9540 con 0x7f118413e3c0 2023-07-29T13:28:02.621 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:02.621 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11840a9a20 con 0x7f1184140da0 2023-07-29T13:28:02.621 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:02.621 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118be42640 10 monclient(hunting): authenticate will time out at 1404.748258s 2023-07-29T13:28:02.621 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.615+0000 7f118a3df640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1184140da0 0x7f1184141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:02.621 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.616+0000 7f118a3df640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1184140da0 0x7f1184141200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:43182/0 (socket says 172.21.15.40:43182) 2023-07-29T13:28:02.622 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.616+0000 7f118a3df640 1 -- 172.21.15.40:0/1316018089 learned_addr learned my addr 172.21.15.40:0/1316018089 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:02.622 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.616+0000 7f118a3df640 10 monclient(hunting): get_auth_request con 0x7f1184140da0 auth_method 0 2023-07-29T13:28:02.622 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.616+0000 7f118a3df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:02.622 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.616+0000 7f118a3df640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:02.622 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.616+0000 7f118a3df640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:02.622 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.616+0000 7f118a3df640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:02.622 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.616+0000 7f118a3df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:02.622 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.616+0000 7f118a3df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:02.623 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.618+0000 7f118a3df640 10 monclient(hunting): handle_auth_done global_id 4190 payload 274 2023-07-29T13:28:02.623 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.618+0000 7f118a3df640 1 -- 172.21.15.40:0/1316018089 >> v2:172.21.15.45:3300/0 conn(0x7f118413e3c0 msgr2=0x7f11840b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:02.623 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.618+0000 7f118a3df640 1 --2- 172.21.15.40:0/1316018089 >> v2:172.21.15.45:3300/0 conn(0x7f118413e3c0 0x7f11840b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:02.624 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.618+0000 7f118a3df640 1 -- 172.21.15.40:0/1316018089 >> v1:172.21.15.40:6789/0 conn(0x7f11840b1a70 legacy=0x7f118413dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:02.624 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.618+0000 7f118a3df640 10 monclient: _finish_hunting 0 2023-07-29T13:28:02.624 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.618+0000 7f118a3df640 1 monclient: found mon.noname-b 2023-07-29T13:28:02.624 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.618+0000 7f118a3df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:02.624 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.618+0000 7f118a3df640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:02.625 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.618+0000 7f118a3df640 1 -- 172.21.15.40:0/1316018089 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11840a9d00 con 0x7f1184140da0 2023-07-29T13:28:02.625 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.620+0000 7f118a3df640 1 --2- 172.21.15.40:0/1316018089 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1184140da0 0x7f1184141200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f117c009e50 tx=0x7f117c02be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54dc1b5d29a1dd86 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:02.625 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 1 -- 172.21.15.40:0/1316018089 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c02c9e0 con 0x7f1184140da0 2023-07-29T13:28:02.626 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:02.626 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:02.626 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 10 monclient: dump: 2023-07-29T13:28:02.626 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:28:02.626 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:02.626 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:02.626 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:02.626 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:02.627 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:02.627 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:02.627 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:02.627 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:02.627 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:02.627 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 10 monclient: _finish_auth 0 2023-07-29T13:28:02.627 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 10 monclient: _check_auth_tickets 2023-07-29T13:28:02.627 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:02.628 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 1 -- 172.21.15.40:0/1316018089 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c02cb40 con 0x7f1184140da0 2023-07-29T13:28:02.628 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:02.628 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 1 -- 172.21.15.40:0/1316018089 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0343f0 con 0x7f1184140da0 2023-07-29T13:28:02.628 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f118be42640 5 monclient: authenticate success, global_id 4190 2023-07-29T13:28:02.628 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f118be42640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:02.628 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f118be42640 4 set_mon_vals no callback set 2023-07-29T13:28:02.628 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:02.629 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:02.629 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.624+0000 7f1188bdc640 10 monclient: dump: 2023-07-29T13:28:02.629 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:28:02.629 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:02.629 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:02.629 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:02.629 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:02.629 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:02.630 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:02.630 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:02.630 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:02.630 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:02.632 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 10 monclient: shutdown 2023-07-29T13:28:02.632 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:02.633 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 1 -- 172.21.15.40:0/1316018089 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1184140da0 msgr2=0x7f1184141200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:02.633 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 1 --2- 172.21.15.40:0/1316018089 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1184140da0 0x7f1184141200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f117c009e50 tx=0x7f117c02be40 comp rx=0 tx=0).stop 2023-07-29T13:28:02.633 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 1 -- 172.21.15.40:0/1316018089 shutdown_connections 2023-07-29T13:28:02.633 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 1 --2- 172.21.15.40:0/1316018089 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1184140da0 0x7f1184141200 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:02.633 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 1 --2- 172.21.15.40:0/1316018089 >> v2:172.21.15.45:3300/0 conn(0x7f118413e3c0 0x7f11840b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:02.633 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 1 -- 172.21.15.40:0/1316018089 >> 172.21.15.40:0/1316018089 conn(0x7f1184071680 msgr2=0x7f11840659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:02.634 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 1 -- 172.21.15.40:0/1316018089 shutdown_connections 2023-07-29T13:28:02.634 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 1 -- 172.21.15.40:0/1316018089 wait complete. 2023-07-29T13:28:02.634 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 10 monclient: build_initial_monmap 2023-07-29T13:28:02.634 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.625+0000 7f118be42640 10 monclient: monmap: 2023-07-29T13:28:02.634 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:28:02.634 INFO:teuthology.misc.health.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:02.634 INFO:teuthology.misc.health.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:02.634 INFO:teuthology.misc.health.smithi040.stderr:created 0.000000 2023-07-29T13:28:02.635 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:02.635 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:02.635 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:02.635 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:02.635 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:02.635 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:02.636 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.626+0000 7f118be42640 1 librados: starting msgr at 2023-07-29T13:28:02.636 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.626+0000 7f118be42640 1 librados: starting objecter 2023-07-29T13:28:02.636 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.626+0000 7f118be42640 1 Processor -- start 2023-07-29T13:28:02.636 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.626+0000 7f118be42640 1 -- start start 2023-07-29T13:28:02.636 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.626+0000 7f118be42640 1 librados: setting wanted keys 2023-07-29T13:28:02.636 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.626+0000 7f118be42640 1 librados: calling monclient init 2023-07-29T13:28:02.636 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.626+0000 7f118be42640 10 monclient: init 2023-07-29T13:28:02.637 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.626+0000 7f118be42640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:02.637 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T13:28:02.637 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f11840b1a70 0x7f118413fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:02.637 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 10 monclient(hunting): picked mon.noname-c con 0x7f11840b1a70 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:02.637 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f118413e3c0 0x7f11841d92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:02.637 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 10 monclient(hunting): picked mon.noname-a con 0x7f118413e3c0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:02.638 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1184140da0 0x7f11841d9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:02.638 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 10 monclient(hunting): picked mon.noname-b con 0x7f1184140da0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:02.638 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:02.638 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1184144750 con 0x7f118413e3c0 2023-07-29T13:28:02.638 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:02.638 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1184143880 con 0x7f1184140da0 2023-07-29T13:28:02.639 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:02.639 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1184140930 con 0x7f11840b1a70 2023-07-29T13:28:02.639 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:02.639 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f118be42640 10 monclient(hunting): authenticate will time out at 1404.760182s 2023-07-29T13:28:02.639 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.627+0000 7f1189bde640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f11840b1a70 0x7f118413fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:02.640 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.628+0000 7f1189bde640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f11840b1a70 0x7f118413fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:43188/0 (socket says 172.21.15.40:43188) 2023-07-29T13:28:02.640 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.628+0000 7f1189bde640 1 -- 172.21.15.40:0/1855639887 learned_addr learned my addr 172.21.15.40:0/1855639887 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:02.640 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.628+0000 7f1189bde640 10 monclient(hunting): get_auth_request con 0x7f11840b1a70 auth_method 0 2023-07-29T13:28:02.640 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.628+0000 7f1189bde640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:02.640 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.628+0000 7f1189bde640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:02.640 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.628+0000 7f1189bde640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:02.640 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.628+0000 7f1189bde640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:02.641 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.628+0000 7f1189bde640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:02.641 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.628+0000 7f1189bde640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:02.641 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.633+0000 7f1189bde640 10 monclient(hunting): handle_auth_done global_id 4193 payload 824 2023-07-29T13:28:02.642 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.633+0000 7f1189bde640 1 -- 172.21.15.40:0/1855639887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1184140da0 msgr2=0x7f11841d9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:02.642 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.633+0000 7f1189bde640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1184140da0 0x7f11841d9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:02.642 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.633+0000 7f1189bde640 1 -- 172.21.15.40:0/1855639887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f118413e3c0 msgr2=0x7f11841d92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:02.642 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.633+0000 7f1189bde640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f118413e3c0 0x7f11841d92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:02.642 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.633+0000 7f1189bde640 10 monclient: _finish_hunting 0 2023-07-29T13:28:02.643 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.633+0000 7f1189bde640 1 monclient: found mon.noname-c 2023-07-29T13:28:02.643 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.633+0000 7f1189bde640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:02.643 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.633+0000 7f1189bde640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:28:02.643 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.633+0000 7f1189bde640 1 -- 172.21.15.40:0/1855639887 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009b30 con 0x7f11840b1a70 2023-07-29T13:28:02.643 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.636+0000 7f1189bde640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f11840b1a70 0x7f118413fe60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f118000b3e0 tx=0x7f118000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:02.643 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.636+0000 7f117a7fc640 10 client.?.objecter ms_handle_connect 0x7f11840b1a70 2023-07-29T13:28:02.643 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.636+0000 7f117a7fc640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:02.644 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 1 -- 172.21.15.40:0/1855639887 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180011830 con 0x7f11840b1a70 2023-07-29T13:28:02.645 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:02.645 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:02.645 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 10 monclient: dump: 2023-07-29T13:28:02.645 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:28:02.645 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:02.645 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:02.646 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:02.646 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:02.646 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:02.646 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:02.646 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:02.646 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:02.646 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:02.646 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 10 monclient: _finish_auth 0 2023-07-29T13:28:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 10 monclient: _check_auth_tickets 2023-07-29T13:28:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 1 -- 172.21.15.40:0/1855639887 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11800119b0 con 0x7f11840b1a70 2023-07-29T13:28:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f118be42640 5 monclient: authenticate success, global_id 4193 2023-07-29T13:28:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 1 -- 172.21.15.40:0/1855639887 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180011d60 con 0x7f11840b1a70 2023-07-29T13:28:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:02.648 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117a7fc640 10 monclient: dump: 2023-07-29T13:28:02.648 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:28:02.648 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:02.648 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:02.648 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:02.648 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:02.648 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:02.649 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:02.649 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:02.649 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:02.649 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:02.649 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f117bfff640 4 set_mon_vals no callback set 2023-07-29T13:28:02.649 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f118be42640 10 monclient: _renew_subs 2023-07-29T13:28:02.649 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f118be42640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:02.650 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.641+0000 7f118be42640 1 -- 172.21.15.40:0/1855639887 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1184144390 con 0x7f11840b1a70 2023-07-29T13:28:02.650 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.642+0000 7f118be42640 10 client.4193.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:02.650 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.642+0000 7f118be42640 10 monclient: _renew_subs 2023-07-29T13:28:02.650 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.642+0000 7f118be42640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:02.650 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.642+0000 7f118be42640 1 -- 172.21.15.40:0/1855639887 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11841441b0 con 0x7f11840b1a70 2023-07-29T13:28:02.650 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.642+0000 7f118be42640 1 librados: init done 2023-07-29T13:28:02.650 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.643+0000 7f118be42640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:02.651 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.643+0000 7f118be42640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:02.651 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.643+0000 7f118be42640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:02.651 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.643+0000 7f118be42640 1 -- 172.21.15.40:0/1855639887 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11841445b0 con 0x7f11840b1a70 2023-07-29T13:28:02.675 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.674+0000 7f117a7fc640 1 -- 172.21.15.40:0/1855639887 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 6) v1 ==== 88153+0+0 (secure 0 0 0) 0x7f118001a400 con 0x7f11840b1a70 2023-07-29T13:28:02.676 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.674+0000 7f117a7fc640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f116806bf20 0x7f116806e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:02.676 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.674+0000 7f11893dd640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f116806bf20 0x7f116806e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:02.677 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.674+0000 7f11893dd640 10 monclient: get_auth_request con 0x7f116806bf20 auth_method 0 2023-07-29T13:28:02.677 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.675+0000 7f117a7fc640 10 client.4193.objecter ms_handle_connect 0x7f116806bf20 2023-07-29T13:28:02.677 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.675+0000 7f11893dd640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f116806bf20 0x7f116806e3d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f11840aa820 tx=0x7f1174006c90 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:02.681 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.679+0000 7f117a7fc640 1 -- 172.21.15.40:0/1855639887 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 1904+0+0 (secure 0 0 0) 0x7f118005c140 con 0x7f11840b1a70 2023-07-29T13:28:02.681 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.679+0000 7f117a7fc640 10 client.4193.objecter ms_dispatch 0x7f1184000b90 osd_map(13..13 src has 1..13) v4 2023-07-29T13:28:02.681 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.679+0000 7f117a7fc640 3 client.4193.objecter handle_osd_map got epochs [13,13] > 0 2023-07-29T13:28:02.681 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.679+0000 7f117a7fc640 3 client.4193.objecter handle_osd_map decoding full epoch 13 2023-07-29T13:28:02.682 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.679+0000 7f117a7fc640 20 client.4193.objecter dump_active .. 0 homeless 2023-07-29T13:28:02.956 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.954+0000 7f117a7fc640 1 -- 172.21.15.40:0/1855639887 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f11800532c0 con 0x7f11840b1a70 2023-07-29T13:28:02.956 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.954+0000 7f117a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:02.956 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:02.954+0000 7f117a7fc640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:03.211 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.209+0000 7f118be42640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:03.211 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.209+0000 7f118be42640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:03.211 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.209+0000 7f118be42640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:03.211 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.209+0000 7f118be42640 1 -- 172.21.15.40:0/1855639887 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1184144580 con 0x7f11840b1a70 2023-07-29T13:28:03.268 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:28:03.260+0000 c495640 -1 osd.5 0 waiting for initial osdmap 2023-07-29T13:28:03.317 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.315+0000 7f117a7fc640 1 -- 172.21.15.40:0/1855639887 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7494 (secure 0 0 0) 0x7f1180015020 con 0x7f11840b1a70 2023-07-29T13:28:03.317 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.315+0000 7f117a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:03.318 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.315+0000 7f117a7fc640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:03.318 INFO:teuthology.misc.health.smithi040.stdout: 2023-07-29T13:28:03.318 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":14,"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","created":"2023-07-29T13:27:17.185104+0000","modified":"2023-07-29T13:28:02.564792+0000","last_up_change":"2023-07-29T13:28:02.564792+0000","last_in_change":"2023-07-29T13:27:22.831496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6069a944-26b4-4b00-94e2-f91c485e8957","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab18edd4-e1cb-4815-b5a8-af2b4b2e790b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4575bc2c-4d43-46d3-95ba-1dc2c94278e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e7224640-7d33-4bb7-81a5-99fc26da7763","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23cd76f6-788b-48cf-9438-e931659d3a1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0ddb1e3f-0f65-40d6-ab50-5dc521774445","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4b9d2830-9f2c-438c-9c0b-b6692e21f921","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6593807d-34a1-431b-aa23-24e8340d887b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":85182},{"type":"v1","addr":"172.21.15.45:6803","nonce":85182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":85182},{"type":"v1","addr":"172.21.15.45:6805","nonce":85182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":85182},{"type":"v1","addr":"172.21.15.45:6809","nonce":85182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":85182},{"type":"v1","addr":"172.21.15.45:6807","nonce":85182}]},"public_addr":"172.21.15.45:6803/85182","cluster_addr":"172.21.15.45:6805/85182","heartbeat_back_addr":"172.21.15.45:6809/85182","heartbeat_front_addr":"172.21.15.45:6807/85182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-29T13:28:03.320 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 10 librados: watch_flush enter 2023-07-29T13:28:03.320 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 10 librados: watch_flush exit 2023-07-29T13:28:03.321 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 20 client.4193.objecter shutdown clearing up homeless session... 2023-07-29T13:28:03.321 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 10 client.4193.objecter successfully canceled tick 2023-07-29T13:28:03.321 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 -- 172.21.15.40:0/1855639887 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f116806bf20 msgr2=0x7f116806e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:03.321 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f116806bf20 0x7f116806e3d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f11840aa820 tx=0x7f1174006c90 comp rx=0 tx=0).stop 2023-07-29T13:28:03.321 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 10 monclient: shutdown 2023-07-29T13:28:03.321 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:03.321 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 -- 172.21.15.40:0/1855639887 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f11840b1a70 msgr2=0x7f118413fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:03.322 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f11840b1a70 0x7f118413fe60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f118000b3e0 tx=0x7f118000b8a0 comp rx=0 tx=0).stop 2023-07-29T13:28:03.322 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 -- 172.21.15.40:0/1855639887 shutdown_connections 2023-07-29T13:28:03.322 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f116806bf20 0x7f116806e3d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:03.322 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1184140da0 0x7f11841d9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:03.322 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f118413e3c0 0x7f11841d92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:03.322 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 --2- 172.21.15.40:0/1855639887 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f11840b1a70 0x7f118413fe60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:03.322 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 -- 172.21.15.40:0/1855639887 >> 172.21.15.40:0/1855639887 conn(0x7f1184071680 msgr2=0x7f118413dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:03.323 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 -- 172.21.15.40:0/1855639887 shutdown_connections 2023-07-29T13:28:03.323 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 -- 172.21.15.40:0/1855639887 wait complete. 2023-07-29T13:28:03.323 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:03.318+0000 7f118be42640 1 librados: shutdown 2023-07-29T13:28:03.329 DEBUG:teuthology.misc:1 of 8 OSDs are up 2023-07-29T13:28:03.382 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:28:03.370+0000 c495640 -1 osd.6 0 waiting for initial osdmap 2023-07-29T13:28:03.649 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:28:03.635+0000 14264640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:28:03.806 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:28:03.791+0000 14264640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:28:04.627 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:28:04.622+0000 c495640 -1 osd.4 0 waiting for initial osdmap 2023-07-29T13:28:04.683 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:28:04.678+0000 ac7c640 -1 osd.0 0 waiting for initial osdmap 2023-07-29T13:28:04.691 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:28:04.687+0000 ac7c640 -1 osd.1 0 waiting for initial osdmap 2023-07-29T13:28:04.716 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:28:04.710+0000 c495640 -1 osd.3 0 waiting for initial osdmap 2023-07-29T13:28:04.732 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:28:04.724+0000 ac7c640 -1 osd.2 0 waiting for initial osdmap 2023-07-29T13:28:05.041 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:28:05.027+0000 12a61640 -1 osd.4 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:28:05.208 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:28:05.192+0000 14264640 -1 osd.3 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:28:05.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:28:05.208+0000 14264640 -1 osd.0 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:28:05.255 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:28:05.239+0000 14264640 -1 osd.2 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:28:05.265 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:28:05.248+0000 14264640 -1 osd.1 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:28:10.136 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-29T13:28:10.227 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.225+0000 7f64c0250640 10 monclient: get_monmap_and_config 2023-07-29T13:28:10.227 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.225+0000 7f64c0250640 10 monclient: build_initial_monmap 2023-07-29T13:28:10.228 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.225+0000 7f64c0250640 10 monclient: monmap: 2023-07-29T13:28:10.228 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:28:10.228 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:10.228 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:28:10.227083+0000 2023-07-29T13:28:10.228 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:28:10.227083+0000 2023-07-29T13:28:10.228 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:10.229 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.229 INFO:teuthology.misc.health.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:10.229 INFO:teuthology.misc.health.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:10.229 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:10.229 INFO:teuthology.misc.health.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:10.229 INFO:teuthology.misc.health.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:10.229 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:10.231 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.226+0000 7f64c0250640 1 Processor -- start 2023-07-29T13:28:10.231 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.226+0000 7f64c0250640 1 -- start start 2023-07-29T13:28:10.231 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.226+0000 7f64c0250640 10 monclient: init 2023-07-29T13:28:10.231 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:10.231 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 10 monclient: _add_conns ranks=[3,4,0,2,1] 2023-07-29T13:28:10.231 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64b80b1a70 0x7f64b80b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:10.232 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 10 monclient(hunting): picked mon.noname-a con 0x7f64b80b1a70 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:10.232 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f64b813dcb0 0x7f64b813e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:10.232 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 10 monclient(hunting): picked mon.noname-c con 0x7f64b813dcb0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:10.232 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f64b8140da0 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:10.232 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b813e650 con 0x7f64b8140da0 2023-07-29T13:28:10.232 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:10.233 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b80a9540 con 0x7f64b80b1a70 2023-07-29T13:28:10.233 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:10.233 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b80a9740 con 0x7f64b813dcb0 2023-07-29T13:28:10.233 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:10.233 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64c0250640 10 monclient(hunting): authenticate will time out at 1412.360418s 2023-07-29T13:28:10.233 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.227+0000 7f64bdfec640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64b80b1a70 0x7f64b80b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:10.233 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bd7eb640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f64b813dcb0 0x7f64b813e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:10.233 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bd7eb640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f64b813dcb0 0x7f64b813e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:44794/0 (socket says 172.21.15.40:44794) 2023-07-29T13:28:10.234 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bd7eb640 1 -- 172.21.15.40:0/2765040967 learned_addr learned my addr 172.21.15.40:0/2765040967 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:10.234 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bd7eb640 10 monclient(hunting): get_auth_request con 0x7f64b813dcb0 auth_method 0 2023-07-29T13:28:10.234 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bd7eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:10.234 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bd7eb640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:10.234 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bd7eb640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:10.234 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bdfec640 10 monclient(hunting): get_auth_request con 0x7f64b80b1a70 auth_method 0 2023-07-29T13:28:10.234 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bdfec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:10.234 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bdfec640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:10.235 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bdfec640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:10.235 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bd7eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:10.235 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bd7eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:10.235 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.228+0000 7f64bd7eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:10.235 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.229+0000 7f64bdfec640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:10.235 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.229+0000 7f64bdfec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:10.235 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.229+0000 7f64bdfec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:10.236 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.230+0000 7f64bd7eb640 10 monclient(hunting): handle_auth_done global_id 4252 payload 274 2023-07-29T13:28:10.236 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.230+0000 7f64bd7eb640 1 -- 172.21.15.40:0/2765040967 >> v2:172.21.15.40:3300/0 conn(0x7f64b80b1a70 msgr2=0x7f64b80b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:10.236 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.230+0000 7f64bd7eb640 1 --2- 172.21.15.40:0/2765040967 >> v2:172.21.15.40:3300/0 conn(0x7f64b80b1a70 0x7f64b80b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.236 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.230+0000 7f64bd7eb640 1 -- 172.21.15.40:0/2765040967 >> v1:172.21.15.40:6789/0 conn(0x7f64b8140da0 legacy=0x7f64b81411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:10.237 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.230+0000 7f64bd7eb640 10 monclient: _finish_hunting 0 2023-07-29T13:28:10.237 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.230+0000 7f64bd7eb640 1 monclient: found mon.noname-c 2023-07-29T13:28:10.237 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.230+0000 7f64bd7eb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:10.237 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.230+0000 7f64bd7eb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:28:10.237 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.230+0000 7f64bd7eb640 1 -- 172.21.15.40:0/2765040967 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b80a9a20 con 0x7f64b813dcb0 2023-07-29T13:28:10.237 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.232+0000 7f64bd7eb640 1 --2- 172.21.15.40:0/2765040967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b813dcb0 0x7f64b813e110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64b8066610 tx=0x7f64b402b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aee78325073c59 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:10.238 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 1 -- 172.21.15.40:0/2765040967 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b40317e0 con 0x7f64b813dcb0 2023-07-29T13:28:10.238 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:10.239 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:10.239 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 10 monclient: dump: 2023-07-29T13:28:10.239 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:28:10.239 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:10.239 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.239 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.240 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:10.240 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.240 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:10.240 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:10.240 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:10.240 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:10.240 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 10 monclient: _finish_auth 0 2023-07-29T13:28:10.240 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 10 monclient: _check_auth_tickets 2023-07-29T13:28:10.241 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:10.241 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 1 -- 172.21.15.40:0/2765040967 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b4031940 con 0x7f64b813dcb0 2023-07-29T13:28:10.241 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:10.241 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 1 -- 172.21.15.40:0/2765040967 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4031cd0 con 0x7f64b813dcb0 2023-07-29T13:28:10.241 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64c0250640 5 monclient: authenticate success, global_id 4252 2023-07-29T13:28:10.241 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64c0250640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:10.241 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64c0250640 4 set_mon_vals no callback set 2023-07-29T13:28:10.242 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:10.242 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:10.242 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64bcfea640 10 monclient: dump: 2023-07-29T13:28:10.242 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:28:10.242 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:10.242 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.242 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.242 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:10.243 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.243 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:10.243 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:10.243 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:10.243 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:10.243 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64c0250640 10 monclient: shutdown 2023-07-29T13:28:10.243 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64c0250640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:10.243 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64c0250640 1 -- 172.21.15.40:0/2765040967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b813dcb0 msgr2=0x7f64b813e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:10.244 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64c0250640 1 --2- 172.21.15.40:0/2765040967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b813dcb0 0x7f64b813e110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64b8066610 tx=0x7f64b402b880 comp rx=0 tx=0).stop 2023-07-29T13:28:10.244 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64c0250640 1 -- 172.21.15.40:0/2765040967 shutdown_connections 2023-07-29T13:28:10.244 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64c0250640 1 --2- 172.21.15.40:0/2765040967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b813dcb0 0x7f64b813e110 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.244 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64c0250640 1 --2- 172.21.15.40:0/2765040967 >> v2:172.21.15.40:3300/0 conn(0x7f64b80b1a70 0x7f64b80b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.244 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.236+0000 7f64c0250640 1 -- 172.21.15.40:0/2765040967 >> 172.21.15.40:0/2765040967 conn(0x7f64b8071680 msgr2=0x7f64b80659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:10.244 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.237+0000 7f64c0250640 1 -- 172.21.15.40:0/2765040967 shutdown_connections 2023-07-29T13:28:10.244 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.237+0000 7f64c0250640 1 -- 172.21.15.40:0/2765040967 wait complete. 2023-07-29T13:28:10.247 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.237+0000 7f64c0250640 10 monclient: build_initial_monmap 2023-07-29T13:28:10.247 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.237+0000 7f64c0250640 10 monclient: monmap: 2023-07-29T13:28:10.247 INFO:teuthology.misc.health.smithi040.stderr:epoch 0 2023-07-29T13:28:10.247 INFO:teuthology.misc.health.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:10.247 INFO:teuthology.misc.health.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:10.247 INFO:teuthology.misc.health.smithi040.stderr:created 0.000000 2023-07-29T13:28:10.248 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:10.248 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.248 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:10.248 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:10.248 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:10.248 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:10.248 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.237+0000 7f64c0250640 1 librados: starting msgr at 2023-07-29T13:28:10.249 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.237+0000 7f64c0250640 1 librados: starting objecter 2023-07-29T13:28:10.249 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 1 Processor -- start 2023-07-29T13:28:10.249 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 1 -- start start 2023-07-29T13:28:10.249 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 1 librados: setting wanted keys 2023-07-29T13:28:10.249 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 1 librados: calling monclient init 2023-07-29T13:28:10.249 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 10 monclient: init 2023-07-29T13:28:10.249 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:10.249 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 10 monclient: _add_conns ranks=[1,0,2] 2023-07-29T13:28:10.250 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b80b1a70 0x7f64b81d8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:10.250 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 10 monclient(hunting): picked mon.noname-b con 0x7f64b80b1a70 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:10.250 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b813dcb0 0x7f64b80af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:10.250 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 10 monclient(hunting): picked mon.noname-a con 0x7f64b813dcb0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:10.250 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f64b8140da0 0x7f64b80adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:10.250 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 10 monclient(hunting): picked mon.noname-c con 0x7f64b8140da0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:10.250 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:10.251 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b81d9020 con 0x7f64b813dcb0 2023-07-29T13:28:10.251 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:10.251 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80ac350 con 0x7f64b80b1a70 2023-07-29T13:28:10.251 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:10.251 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b81d9200 con 0x7f64b8140da0 2023-07-29T13:28:10.251 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:10.251 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.238+0000 7f64c0250640 10 monclient(hunting): authenticate will time out at 1412.371834s 2023-07-29T13:28:10.251 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bdfec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b80b1a70 0x7f64b81d8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:10.252 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bd7eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b813dcb0 0x7f64b80af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:10.252 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64be7ed640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f64b8140da0 0x7f64b80adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:10.252 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bdfec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b80b1a70 0x7f64b81d8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:44796/0 (socket says 172.21.15.40:44796) 2023-07-29T13:28:10.252 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bdfec640 1 -- 172.21.15.40:0/2617501905 learned_addr learned my addr 172.21.15.40:0/2617501905 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:10.252 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bdfec640 10 monclient(hunting): get_auth_request con 0x7f64b80b1a70 auth_method 0 2023-07-29T13:28:10.252 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bdfec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:10.252 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bdfec640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:10.253 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bdfec640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:10.253 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bd7eb640 10 monclient(hunting): get_auth_request con 0x7f64b813dcb0 auth_method 0 2023-07-29T13:28:10.253 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bd7eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:10.253 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bd7eb640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:10.253 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64bd7eb640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:10.253 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64be7ed640 10 monclient(hunting): get_auth_request con 0x7f64b8140da0 auth_method 0 2023-07-29T13:28:10.253 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64be7ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:10.253 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64be7ed640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:10.254 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.239+0000 7f64be7ed640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:10.254 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.240+0000 7f64bdfec640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:10.254 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.240+0000 7f64bdfec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:10.254 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.240+0000 7f64bdfec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:10.254 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.240+0000 7f64bd7eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:10.254 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.240+0000 7f64bd7eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:10.254 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.240+0000 7f64bd7eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:10.254 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.240+0000 7f64be7ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:10.255 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.240+0000 7f64be7ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:10.255 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.240+0000 7f64be7ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:10.255 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.242+0000 7f64bdfec640 10 monclient(hunting): handle_auth_done global_id 4255 payload 824 2023-07-29T13:28:10.255 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.242+0000 7f64bdfec640 1 -- 172.21.15.40:0/2617501905 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f64b8140da0 msgr2=0x7f64b80adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:10.255 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.242+0000 7f64bdfec640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f64b8140da0 0x7f64b80adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.255 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.242+0000 7f64bdfec640 1 -- 172.21.15.40:0/2617501905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b813dcb0 msgr2=0x7f64b80af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:10.255 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.242+0000 7f64bdfec640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b813dcb0 0x7f64b80af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.255 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.242+0000 7f64bdfec640 10 monclient: _finish_hunting 0 2023-07-29T13:28:10.256 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.242+0000 7f64bdfec640 1 monclient: found mon.noname-b 2023-07-29T13:28:10.256 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.242+0000 7f64bdfec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:10.256 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.242+0000 7f64bdfec640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:10.256 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.242+0000 7f64bdfec640 1 -- 172.21.15.40:0/2617501905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b40094d0 con 0x7f64b80b1a70 2023-07-29T13:28:10.256 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.245+0000 7f64bdfec640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b80b1a70 0x7f64b81d8ae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64a800e9b0 tx=0x7f64a800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:10.256 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.245+0000 7f64ae7fc640 10 client.?.objecter ms_handle_connect 0x7f64b80b1a70 2023-07-29T13:28:10.256 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.245+0000 7f64ae7fc640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:10.258 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 1 -- 172.21.15.40:0/2617501905 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8009670 con 0x7f64b80b1a70 2023-07-29T13:28:10.258 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:10.258 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:10.258 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 10 monclient: dump: 2023-07-29T13:28:10.258 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:28:10.258 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:10.259 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.259 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.259 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:10.259 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.259 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:10.259 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:10.259 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:10.260 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:10.260 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 10 monclient: _finish_auth 0 2023-07-29T13:28:10.260 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 10 monclient: _check_auth_tickets 2023-07-29T13:28:10.260 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:10.260 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 1 -- 172.21.15.40:0/2617501905 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a800c9a0 con 0x7f64b80b1a70 2023-07-29T13:28:10.260 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64c0250640 5 monclient: authenticate success, global_id 4255 2023-07-29T13:28:10.260 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64c0250640 10 monclient: _renew_subs 2023-07-29T13:28:10.261 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64c0250640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:10.261 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64c0250640 1 -- 172.21.15.40:0/2617501905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b80afe30 con 0x7f64b80b1a70 2023-07-29T13:28:10.261 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:10.261 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 1 -- 172.21.15.40:0/2617501905 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a800cd50 con 0x7f64b80b1a70 2023-07-29T13:28:10.261 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:10.261 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:10.261 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64ae7fc640 10 monclient: dump: 2023-07-29T13:28:10.261 INFO:teuthology.misc.health.smithi040.stderr:epoch 1 2023-07-29T13:28:10.262 INFO:teuthology.misc.health.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:10.262 INFO:teuthology.misc.health.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.262 INFO:teuthology.misc.health.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.262 INFO:teuthology.misc.health.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:10.262 INFO:teuthology.misc.health.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.262 INFO:teuthology.misc.health.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:10.262 INFO:teuthology.misc.health.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:10.262 INFO:teuthology.misc.health.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:10.263 INFO:teuthology.misc.health.smithi040.stderr: 2023-07-29T13:28:10.263 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64affff640 4 set_mon_vals no callback set 2023-07-29T13:28:10.263 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64c0250640 10 client.4255.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:10.263 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64c0250640 10 monclient: _renew_subs 2023-07-29T13:28:10.263 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64c0250640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:10.263 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64c0250640 1 -- 172.21.15.40:0/2617501905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b813baa0 con 0x7f64b80b1a70 2023-07-29T13:28:10.263 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.248+0000 7f64c0250640 1 librados: init done 2023-07-29T13:28:10.264 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.249+0000 7f64c0250640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:10.264 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.249+0000 7f64c0250640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:10.264 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.249+0000 7f64c0250640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:10.264 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.249+0000 7f64c0250640 1 -- 172.21.15.40:0/2617501905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b81d93c0 con 0x7f64b80b1a70 2023-07-29T13:28:10.269 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.268+0000 7f64ae7fc640 1 -- 172.21.15.40:0/2617501905 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 6) v1 ==== 88153+0+0 (secure 0 0 0) 0x7f64a8013470 con 0x7f64b80b1a70 2023-07-29T13:28:10.270 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.268+0000 7f64ae7fc640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f649c06bf20 0x7f649c06e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:10.270 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.268+0000 7f64bd7eb640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f649c06bf20 0x7f649c06e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:10.270 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.268+0000 7f64bd7eb640 10 monclient: get_auth_request con 0x7f649c06bf20 auth_method 0 2023-07-29T13:28:10.270 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.269+0000 7f64bd7eb640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f649c06bf20 0x7f649c06e3d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f64b4036000 tx=0x7f64b403d040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:10.271 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.269+0000 7f64ae7fc640 10 client.4255.objecter ms_handle_connect 0x7f649c06bf20 2023-07-29T13:28:10.271 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.270+0000 7f64ae7fc640 1 -- 172.21.15.40:0/2617501905 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f64a805c4f0 con 0x7f64b80b1a70 2023-07-29T13:28:10.272 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.270+0000 7f64ae7fc640 10 client.4255.objecter ms_dispatch 0x7f64b8000b90 osd_map(19..19 src has 1..19) v4 2023-07-29T13:28:10.272 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.270+0000 7f64ae7fc640 3 client.4255.objecter handle_osd_map got epochs [19,19] > 0 2023-07-29T13:28:10.272 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.270+0000 7f64ae7fc640 3 client.4255.objecter handle_osd_map decoding full epoch 19 2023-07-29T13:28:10.272 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.270+0000 7f64ae7fc640 20 client.4255.objecter dump_active .. 0 homeless 2023-07-29T13:28:10.528 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.527+0000 7f64ae7fc640 1 -- 172.21.15.40:0/2617501905 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f64a8054920 con 0x7f64b80b1a70 2023-07-29T13:28:10.529 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.527+0000 7f64ae7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:10.529 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.527+0000 7f64ae7fc640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:10.721 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.719+0000 7f64c0250640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:10.722 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.719+0000 7f64c0250640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:10.722 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.719+0000 7f64c0250640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:10.722 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.719+0000 7f64c0250640 1 -- 172.21.15.40:0/2617501905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64b8069be0 con 0x7f64b80b1a70 2023-07-29T13:28:10.829 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.827+0000 7f64ae7fc640 1 -- 172.21.15.40:0/2617501905 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12347 (secure 0 0 0) 0x7f64a8017090 con 0x7f64b80b1a70 2023-07-29T13:28:10.829 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.827+0000 7f64ae7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:10.830 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.827+0000 7f64ae7fc640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:10.830 INFO:teuthology.misc.health.smithi040.stdout: 2023-07-29T13:28:10.830 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":19,"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","created":"2023-07-29T13:27:17.185104+0000","modified":"2023-07-29T13:28:08.798778+0000","last_up_change":"2023-07-29T13:28:06.434639+0000","last_in_change":"2023-07-29T13:27:22.831496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-29T13:28:05.611243+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6069a944-26b4-4b00-94e2-f91c485e8957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":85570},{"type":"v1","addr":"172.21.15.40:6809","nonce":85570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":85570},{"type":"v1","addr":"172.21.15.40:6811","nonce":85570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":85570},{"type":"v1","addr":"172.21.15.40:6815","nonce":85570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":85570},{"type":"v1","addr":"172.21.15.40:6813","nonce":85570}]},"public_addr":"172.21.15.40:6809/85570","cluster_addr":"172.21.15.40:6811/85570","heartbeat_back_addr":"172.21.15.40:6815/85570","heartbeat_front_addr":"172.21.15.40:6813/85570","state":["exists","up"]},{"osd":1,"uuid":"ab18edd4-e1cb-4815-b5a8-af2b4b2e790b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":85573},{"type":"v1","addr":"172.21.15.40:6817","nonce":85573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":85573},{"type":"v1","addr":"172.21.15.40:6819","nonce":85573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":85573},{"type":"v1","addr":"172.21.15.40:6823","nonce":85573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":85573},{"type":"v1","addr":"172.21.15.40:6821","nonce":85573}]},"public_addr":"172.21.15.40:6817/85573","cluster_addr":"172.21.15.40:6819/85573","heartbeat_back_addr":"172.21.15.40:6823/85573","heartbeat_front_addr":"172.21.15.40:6821/85573","state":["exists","up"]},{"osd":2,"uuid":"4575bc2c-4d43-46d3-95ba-1dc2c94278e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":85572},{"type":"v1","addr":"172.21.15.40:6801","nonce":85572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":85572},{"type":"v1","addr":"172.21.15.40:6803","nonce":85572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":85572},{"type":"v1","addr":"172.21.15.40:6807","nonce":85572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":85572},{"type":"v1","addr":"172.21.15.40:6805","nonce":85572}]},"public_addr":"172.21.15.40:6801/85572","cluster_addr":"172.21.15.40:6803/85572","heartbeat_back_addr":"172.21.15.40:6807/85572","heartbeat_front_addr":"172.21.15.40:6805/85572","state":["exists","up"]},{"osd":3,"uuid":"e7224640-7d33-4bb7-81a5-99fc26da7763","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":85571},{"type":"v1","addr":"172.21.15.40:6825","nonce":85571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":85571},{"type":"v1","addr":"172.21.15.40:6827","nonce":85571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":85571},{"type":"v1","addr":"172.21.15.40:6831","nonce":85571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":85571},{"type":"v1","addr":"172.21.15.40:6829","nonce":85571}]},"public_addr":"172.21.15.40:6825/85571","cluster_addr":"172.21.15.40:6827/85571","heartbeat_back_addr":"172.21.15.40:6831/85571","heartbeat_front_addr":"172.21.15.40:6829/85571","state":["exists","up"]},{"osd":4,"uuid":"23cd76f6-788b-48cf-9438-e931659d3a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":85183},{"type":"v1","addr":"172.21.15.45:6827","nonce":85183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":85183},{"type":"v1","addr":"172.21.15.45:6829","nonce":85183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":85183},{"type":"v1","addr":"172.21.15.45:6833","nonce":85183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":85183},{"type":"v1","addr":"172.21.15.45:6831","nonce":85183}]},"public_addr":"172.21.15.45:6827/85183","cluster_addr":"172.21.15.45:6829/85183","heartbeat_back_addr":"172.21.15.45:6833/85183","heartbeat_front_addr":"172.21.15.45:6831/85183","state":["exists","up"]},{"osd":5,"uuid":"0ddb1e3f-0f65-40d6-ab50-5dc521774445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":85181},{"type":"v1","addr":"172.21.15.45:6813","nonce":85181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":85181},{"type":"v1","addr":"172.21.15.45:6818","nonce":85181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6823","nonce":85181},{"type":"v1","addr":"172.21.15.45:6825","nonce":85181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":85181},{"type":"v1","addr":"172.21.15.45:6821","nonce":85181}]},"public_addr":"172.21.15.45:6813/85181","cluster_addr":"172.21.15.45:6818/85181","heartbeat_back_addr":"172.21.15.45:6825/85181","heartbeat_front_addr":"172.21.15.45:6821/85181","state":["exists","up"]},{"osd":6,"uuid":"4b9d2830-9f2c-438c-9c0b-b6692e21f921","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":85184},{"type":"v1","addr":"172.21.15.45:6811","nonce":85184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":85184},{"type":"v1","addr":"172.21.15.45:6815","nonce":85184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":85184},{"type":"v1","addr":"172.21.15.45:6824","nonce":85184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":85184},{"type":"v1","addr":"172.21.15.45:6819","nonce":85184}]},"public_addr":"172.21.15.45:6811/85184","cluster_addr":"172.21.15.45:6815/85184","heartbeat_back_addr":"172.21.15.45:6824/85184","heartbeat_front_addr":"172.21.15.45:6819/85184","state":["exists","up"]},{"osd":7,"uuid":"6593807d-34a1-431b-aa23-24e8340d887b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":85182},{"type":"v1","addr":"172.21.15.45:6803","nonce":85182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":85182},{"type":"v1","addr":"172.21.15.45:6805","nonce":85182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":85182},{"type":"v1","addr":"172.21.15.45:6809","nonce":85182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":85182},{"type":"v1","addr":"172.21.15.45:6807","nonce":85182}]},"public_addr":"172.21.15.45:6803/85182","cluster_addr":"172.21.15.45:6805/85182","heartbeat_back_addr":"172.21.15.45:6809/85182","heartbeat_front_addr":"172.21.15.45:6807/85182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.729512+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.651149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.683349+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:03.275000+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:03.102406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:01.537435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:01.740924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:00.468651+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-29T13:28:10.832 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 10 librados: watch_flush enter 2023-07-29T13:28:10.832 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 10 librados: watch_flush exit 2023-07-29T13:28:10.832 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 20 client.4255.objecter shutdown clearing up homeless session... 2023-07-29T13:28:10.832 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 10 client.4255.objecter successfully canceled tick 2023-07-29T13:28:10.832 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 -- 172.21.15.40:0/2617501905 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f649c06bf20 msgr2=0x7f649c06e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:10.833 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f649c06bf20 0x7f649c06e3d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f64b4036000 tx=0x7f64b403d040 comp rx=0 tx=0).stop 2023-07-29T13:28:10.833 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 10 monclient: shutdown 2023-07-29T13:28:10.833 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:10.833 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 -- 172.21.15.40:0/2617501905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b80b1a70 msgr2=0x7f64b81d8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:10.833 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b80b1a70 0x7f64b81d8ae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64a800e9b0 tx=0x7f64a800ee70 comp rx=0 tx=0).stop 2023-07-29T13:28:10.833 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 -- 172.21.15.40:0/2617501905 shutdown_connections 2023-07-29T13:28:10.833 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f649c06bf20 0x7f649c06e3d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.833 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f64b8140da0 0x7f64b80adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.834 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b813dcb0 0x7f64b80af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.834 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 --2- 172.21.15.40:0/2617501905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b80b1a70 0x7f64b81d8ae0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.834 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 -- 172.21.15.40:0/2617501905 >> 172.21.15.40:0/2617501905 conn(0x7f64b8071680 msgr2=0x7f64b813d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:10.834 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 -- 172.21.15.40:0/2617501905 shutdown_connections 2023-07-29T13:28:10.834 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 -- 172.21.15.40:0/2617501905 wait complete. 2023-07-29T13:28:10.834 INFO:teuthology.misc.health.smithi040.stderr:2023-07-29T13:28:10.830+0000 7f64c0250640 1 librados: shutdown 2023-07-29T13:28:10.842 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-07-29T13:28:10.842 INFO:tasks.ceph:Creating RBD pool 2023-07-29T13:28:10.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd pool create rbd 8 2023-07-29T13:28:10.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.954+0000 7f2cbadf9640 10 monclient: get_monmap_and_config 2023-07-29T13:28:10.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.954+0000 7f2cbadf9640 10 monclient: build_initial_monmap 2023-07-29T13:28:10.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.954+0000 7f2cbadf9640 10 monclient: monmap: 2023-07-29T13:28:10.956 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:10.956 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:10.956 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:10.955280+0000 2023-07-29T13:28:10.956 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:10.955280+0000 2023-07-29T13:28:10.956 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:10.957 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.957 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:10.957 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:10.957 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:10.957 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:10.957 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:10.958 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:10.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.954+0000 7f2cbadf9640 1 Processor -- start 2023-07-29T13:28:10.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.954+0000 7f2cbadf9640 1 -- start start 2023-07-29T13:28:10.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.954+0000 7f2cbadf9640 10 monclient: init 2023-07-29T13:28:10.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:10.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 10 monclient: _add_conns ranks=[0,2,1,4,3] 2023-07-29T13:28:10.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2cb40b3ae0 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:10.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cb413e3a0 0x7f2cb4140c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:10.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 10 monclient(hunting): picked mon.noname-b con 0x7f2cb413e3a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:10.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2cb40a8910 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:10.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb40b3eb0 con 0x7f2cb40b3ae0 2023-07-29T13:28:10.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb40a9950 con 0x7f2cb40a8910 2023-07-29T13:28:10.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:10.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40b0f10 con 0x7f2cb413e3a0 2023-07-29T13:28:10.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:10.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.955+0000 7f2cbadf9640 10 monclient(hunting): authenticate will time out at 1413.088687s 2023-07-29T13:28:10.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.956+0000 7f2cb3fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cb413e3a0 0x7f2cb4140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:10.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.956+0000 7f2cb8b95640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2cb40b3ae0 0x7f2cb413dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51214/0 (socket says 172.21.15.40:51214) 2023-07-29T13:28:10.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.956+0000 7f2cb8b95640 1 -- 172.21.15.40:0/113490820 learned_addr learned my addr 172.21.15.40:0/113490820 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:28:10.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.956+0000 7f2cb3fff640 10 monclient(hunting): get_auth_request con 0x7f2cb413e3a0 auth_method 0 2023-07-29T13:28:10.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.956+0000 7f2cb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:10.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.956+0000 7f2cb3fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:10.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.956+0000 7f2cb3fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:10.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.957+0000 7f2cb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:10.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.957+0000 7f2cb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:10.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.957+0000 7f2cb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:10.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.958+0000 7f2cb3fff640 10 monclient(hunting): handle_auth_done global_id 4199 payload 274 2023-07-29T13:28:10.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.958+0000 7f2cb3fff640 1 -- 172.21.15.40:0/113490820 >> v1:172.21.15.45:6789/0 conn(0x7f2cb40a8910 legacy=0x7f2cb414b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:10.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.958+0000 7f2cb3fff640 1 -- 172.21.15.40:0/113490820 >> v1:172.21.15.40:6789/0 conn(0x7f2cb40b3ae0 legacy=0x7f2cb413dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:10.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.958+0000 7f2cb3fff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:10.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.958+0000 7f2cb3fff640 1 monclient: found mon.noname-b 2023-07-29T13:28:10.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.958+0000 7f2cb3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:10.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.958+0000 7f2cb3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:10.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.958+0000 7f2cb3fff640 1 -- 172.21.15.40:0/113490820 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb40a9c30 con 0x7f2cb413e3a0 2023-07-29T13:28:10.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.963+0000 7f2cb3fff640 1 --2- 172.21.15.40:0/113490820 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cb413e3a0 0x7f2cb4140c10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00097d0 tx=0x7f2ca002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de752a6dc2980279 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:10.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 1 -- 172.21.15.40:0/113490820 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0031800 con 0x7f2cb413e3a0 2023-07-29T13:28:10.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:10.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:10.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 10 monclient: dump: 2023-07-29T13:28:10.969 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:10.969 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:10.969 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.969 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:10.970 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 10 monclient: _finish_auth 0 2023-07-29T13:28:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 10 monclient: _check_auth_tickets 2023-07-29T13:28:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 1 -- 172.21.15.40:0/113490820 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca0031960 con 0x7f2cb413e3a0 2023-07-29T13:28:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 5 monclient: authenticate success, global_id 4199 2023-07-29T13:28:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 1 -- 172.21.15.40:0/113490820 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0031cf0 con 0x7f2cb413e3a0 2023-07-29T13:28:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 4 set_mon_vals no callback set 2023-07-29T13:28:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cb37fe640 10 monclient: dump: 2023-07-29T13:28:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:10.973 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 10 monclient: shutdown 2023-07-29T13:28:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 1 -- 172.21.15.40:0/113490820 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cb413e3a0 msgr2=0x7f2cb4140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 1 --2- 172.21.15.40:0/113490820 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cb413e3a0 0x7f2cb4140c10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00097d0 tx=0x7f2ca002b8a0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 1 -- 172.21.15.40:0/113490820 shutdown_connections 2023-07-29T13:28:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 1 --2- 172.21.15.40:0/113490820 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cb413e3a0 0x7f2cb4140c10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 1 -- 172.21.15.40:0/113490820 >> 172.21.15.40:0/113490820 conn(0x7f2cb4070fc0 msgr2=0x7f2cb40713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.967+0000 7f2cbadf9640 1 -- 172.21.15.40:0/113490820 shutdown_connections 2023-07-29T13:28:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.968+0000 7f2cbadf9640 1 -- 172.21.15.40:0/113490820 wait complete. 2023-07-29T13:28:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.968+0000 7f2cbadf9640 10 monclient: build_initial_monmap 2023-07-29T13:28:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.968+0000 7f2cbadf9640 10 monclient: monmap: 2023-07-29T13:28:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:10.978 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:10.978 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:10.978 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:10.978 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:10.978 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.978 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:10.979 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.969+0000 7f2cbadf9640 1 librados: starting msgr at 2023-07-29T13:28:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.969+0000 7f2cbadf9640 1 librados: starting objecter 2023-07-29T13:28:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.969+0000 7f2cbadf9640 1 Processor -- start 2023-07-29T13:28:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.969+0000 7f2cbadf9640 1 -- start start 2023-07-29T13:28:10.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.969+0000 7f2cbadf9640 1 librados: setting wanted keys 2023-07-29T13:28:10.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.969+0000 7f2cbadf9640 1 librados: calling monclient init 2023-07-29T13:28:10.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.969+0000 7f2cbadf9640 10 monclient: init 2023-07-29T13:28:10.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:10.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:28:10.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cb40a8910 0x7f2cb40b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 10 monclient(hunting): picked mon.noname-c con 0x7f2cb40a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb40b3ae0 0x7f2cb4143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 10 monclient(hunting): picked mon.noname-b con 0x7f2cb40b3ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cb413e3a0 0x7f2cb41d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 10 monclient(hunting): picked mon.noname-a con 0x7f2cb413e3a0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb4144820 con 0x7f2cb413e3a0 2023-07-29T13:28:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb4144cb0 con 0x7f2cb40b3ae0 2023-07-29T13:28:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40b7aa0 con 0x7f2cb40a8910 2023-07-29T13:28:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cbadf9640 10 monclient(hunting): authenticate will time out at 1413.103350s 2023-07-29T13:28:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cb3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb40b3ae0 0x7f2cb4143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.970+0000 7f2cb8b95640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cb40a8910 0x7f2cb40b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb40b3ae0 0x7f2cb4143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:42322/0 (socket says 172.21.15.40:42322) 2023-07-29T13:28:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb3fff640 1 -- 172.21.15.40:0/2316375922 learned_addr learned my addr 172.21.15.40:0/2316375922 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb3fff640 10 monclient(hunting): get_auth_request con 0x7f2cb40b3ae0 auth_method 0 2023-07-29T13:28:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb3fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb3fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb8b95640 10 monclient(hunting): get_auth_request con 0x7f2cb40a8910 auth_method 0 2023-07-29T13:28:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb8b95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb8b95640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb8b95640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb8b95640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb8b95640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.971+0000 7f2cb8b95640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.973+0000 7f2cb9396640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cb413e3a0 0x7f2cb41d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb9396640 10 monclient(hunting): get_auth_request con 0x7f2cb413e3a0 auth_method 0 2023-07-29T13:28:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb9396640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb9396640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb9396640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb3fff640 10 monclient(hunting): handle_auth_done global_id 4261 payload 824 2023-07-29T13:28:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb3fff640 1 -- 172.21.15.40:0/2316375922 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cb40a8910 msgr2=0x7f2cb40b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb3fff640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cb40a8910 0x7f2cb40b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb3fff640 1 -- 172.21.15.40:0/2316375922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cb413e3a0 msgr2=0x7f2cb41d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb3fff640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cb413e3a0 0x7f2cb41d5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb3fff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb3fff640 1 monclient: found mon.noname-b 2023-07-29T13:28:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:10.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.974+0000 7f2cb3fff640 1 -- 172.21.15.40:0/2316375922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca0009400 con 0x7f2cb40b3ae0 2023-07-29T13:28:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.977+0000 7f2cb3fff640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb40b3ae0 0x7f2cb4143c90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00098f0 tx=0x7f2ca00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.977+0000 7f2cb17fa640 10 client.?.objecter ms_handle_connect 0x7f2cb40b3ae0 2023-07-29T13:28:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.977+0000 7f2cb17fa640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 1 -- 172.21.15.40:0/2316375922 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca003bcc0 con 0x7f2cb40b3ae0 2023-07-29T13:28:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 10 monclient: dump: 2023-07-29T13:28:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.990 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:10.990 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:10.990 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:10.990 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:10.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 10 monclient: _finish_auth 0 2023-07-29T13:28:10.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 10 monclient: _check_auth_tickets 2023-07-29T13:28:10.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:10.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cbadf9640 5 monclient: authenticate success, global_id 4261 2023-07-29T13:28:10.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 1 -- 172.21.15.40:0/2316375922 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca003be60 con 0x7f2cb40b3ae0 2023-07-29T13:28:10.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:10.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 1 -- 172.21.15.40:0/2316375922 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0045630 con 0x7f2cb40b3ae0 2023-07-29T13:28:10.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:10.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:10.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb17fa640 10 monclient: dump: 2023-07-29T13:28:10.991 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:10.991 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:10.993 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cb2ffd640 4 set_mon_vals no callback set 2023-07-29T13:28:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cbadf9640 10 monclient: _renew_subs 2023-07-29T13:28:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cbadf9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cbadf9640 1 -- 172.21.15.40:0/2316375922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb4144270 con 0x7f2cb40b3ae0 2023-07-29T13:28:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cbadf9640 10 client.4261.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cbadf9640 10 monclient: _renew_subs 2023-07-29T13:28:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cbadf9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cbadf9640 1 -- 172.21.15.40:0/2316375922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb41d6340 con 0x7f2cb40b3ae0 2023-07-29T13:28:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.981+0000 7f2cbadf9640 1 librados: init done 2023-07-29T13:28:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.982+0000 7f2cbadf9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.982+0000 7f2cbadf9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.982+0000 7f2cbadf9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:10.982+0000 7f2cbadf9640 1 -- 172.21.15.40:0/2316375922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb406bbb0 con 0x7f2cb40b3ae0 2023-07-29T13:28:11.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.236+0000 7f2cb17fa640 1 -- 172.21.15.40:0/2316375922 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 6) v1 ==== 88153+0+0 (secure 0 0 0) 0x7f2ca00443a0 con 0x7f2cb40b3ae0 2023-07-29T13:28:11.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.237+0000 7f2cb17fa640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2c9806bf20 0x7f2c9806e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:11.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.237+0000 7f2cb8b95640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2c9806bf20 0x7f2c9806e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:11.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.237+0000 7f2cb8b95640 10 monclient: get_auth_request con 0x7f2c9806bf20 auth_method 0 2023-07-29T13:28:11.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.237+0000 7f2cb17fa640 10 client.4261.objecter ms_handle_connect 0x7f2c9806bf20 2023-07-29T13:28:11.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.237+0000 7f2cb8b95640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2c9806bf20 0x7f2c9806e3d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8005880 tx=0x7f2ca8009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.241+0000 7f2cb17fa640 1 -- 172.21.15.40:0/2316375922 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2ca002e070 con 0x7f2cb40b3ae0 2023-07-29T13:28:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.241+0000 7f2cb17fa640 10 client.4261.objecter ms_dispatch 0x7f2cb4000b90 osd_map(19..19 src has 1..19) v4 2023-07-29T13:28:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.241+0000 7f2cb17fa640 3 client.4261.objecter handle_osd_map got epochs [19,19] > 0 2023-07-29T13:28:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.241+0000 7f2cb17fa640 3 client.4261.objecter handle_osd_map decoding full epoch 19 2023-07-29T13:28:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.241+0000 7f2cb17fa640 20 client.4261.objecter dump_active .. 0 homeless 2023-07-29T13:28:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.253+0000 7f2cb17fa640 1 -- 172.21.15.40:0/2316375922 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f2ca0076800 con 0x7f2cb40b3ae0 2023-07-29T13:28:11.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.253+0000 7f2cb17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:11.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.253+0000 7f2cb17fa640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:11.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.527+0000 7f2cbadf9640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2023-07-29T13:28:11.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.527+0000 7f2cbadf9640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2023-07-29T13:28:11.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.527+0000 7f2cbadf9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:11.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:11.527+0000 7f2cbadf9640 1 -- 172.21.15.40:0/2316375922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f2cb4140610 con 0x7f2cb40b3ae0 2023-07-29T13:28:12.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.485+0000 7f2cb17fa640 1 -- 172.21.15.40:0/2316375922 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v21) v1 ==== 109+0+0 (secure 0 0 0) 0x7f2ca0035020 con 0x7f2cb40b3ae0 2023-07-29T13:28:12.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.485+0000 7f2cb17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2023-07-29T13:28:12.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.485+0000 7f2cb17fa640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2023-07-29T13:28:12.487 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'rbd' created 2023-07-29T13:28:12.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 10 librados: watch_flush enter 2023-07-29T13:28:12.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 10 librados: watch_flush exit 2023-07-29T13:28:12.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 20 client.4261.objecter shutdown clearing up homeless session... 2023-07-29T13:28:12.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 10 client.4261.objecter successfully canceled tick 2023-07-29T13:28:12.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 -- 172.21.15.40:0/2316375922 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2c9806bf20 msgr2=0x7f2c9806e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:12.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2c9806bf20 0x7f2c9806e3d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8005880 tx=0x7f2ca8009040 comp rx=0 tx=0).stop 2023-07-29T13:28:12.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 10 monclient: shutdown 2023-07-29T13:28:12.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:12.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 -- 172.21.15.40:0/2316375922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb40b3ae0 msgr2=0x7f2cb4143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:12.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb40b3ae0 0x7f2cb4143c90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00098f0 tx=0x7f2ca00046c0 comp rx=0 tx=0).stop 2023-07-29T13:28:12.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 -- 172.21.15.40:0/2316375922 shutdown_connections 2023-07-29T13:28:12.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2c9806bf20 0x7f2c9806e3d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:12.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb40b3ae0 0x7f2cb4143c90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:12.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cb413e3a0 0x7f2cb41d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:12.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 --2- 172.21.15.40:0/2316375922 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cb40a8910 0x7f2cb40b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:12.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 -- 172.21.15.40:0/2316375922 >> 172.21.15.40:0/2316375922 conn(0x7f2cb4070fc0 msgr2=0x7f2cb413fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:12.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 -- 172.21.15.40:0/2316375922 shutdown_connections 2023-07-29T13:28:12.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 -- 172.21.15.40:0/2316375922 wait complete. 2023-07-29T13:28:12.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.487+0000 7f2cbadf9640 1 librados: shutdown 2023-07-29T13:28:12.501 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-07-29T13:28:12.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.626+0000 7f7b267c4640 10 monclient: get_monmap_and_config 2023-07-29T13:28:12.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.626+0000 7f7b267c4640 10 monclient: build_initial_monmap 2023-07-29T13:28:12.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.626+0000 7f7b267c4640 10 monclient: monmap: 2023-07-29T13:28:12.628 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:12.628 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:12.628 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:12.627287+0000 2023-07-29T13:28:12.628 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:12.627287+0000 2023-07-29T13:28:12.629 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:12.629 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:12.629 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:12.629 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:12.629 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:12.629 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:12.629 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:12.630 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:12.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.626+0000 7f7b267c4640 1 Processor -- start 2023-07-29T13:28:12.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.626+0000 7f7b267c4640 1 -- start start 2023-07-29T13:28:12.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.626+0000 7f7b267c4640 10 monclient: init 2023-07-29T13:28:12.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:12.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2023-07-29T13:28:12.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7b200b7760 0x7f7b2013dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:12.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 10 monclient(hunting): picked mon.noname-b con 0x7f7b200b7760 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:12.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7b2013e2c0 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:12.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b20144a40 0x7f7b2014b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:12.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 10 monclient(hunting): picked mon.noname-a con 0x7f7b20144a40 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:12.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b200b7b30 con 0x7f7b2013e2c0 2023-07-29T13:28:12.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:12.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b200b2eb0 con 0x7f7b20144a40 2023-07-29T13:28:12.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:12.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2013e6f0 con 0x7f7b200b7760 2023-07-29T13:28:12.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:12.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.627+0000 7f7b267c4640 10 monclient(hunting): authenticate will time out at 1414.760806s 2023-07-29T13:28:12.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.628+0000 7f7b1f7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7b2013e2c0 0x7f7b20144330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.40:46222/0 (socket says 172.21.15.40:46222) 2023-07-29T13:28:12.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.628+0000 7f7b1f7fe640 1 -- 172.21.15.40:0/3366500681 learned_addr learned my addr 172.21.15.40:0/3366500681 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:28:12.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.628+0000 7f7b24d61640 1 --2- 172.21.15.40:0/3366500681 >> v2:172.21.15.40:3300/0 conn(0x7f7b20144a40 0x7f7b2014b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:12.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.629+0000 7f7b24d61640 10 monclient(hunting): get_auth_request con 0x7f7b20144a40 auth_method 0 2023-07-29T13:28:12.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.629+0000 7f7b24d61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:12.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.629+0000 7f7b24d61640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:12.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.629+0000 7f7b24d61640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:12.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.629+0000 7f7b24d61640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:12.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.629+0000 7f7b24d61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:12.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.629+0000 7f7b24d61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:12.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.630+0000 7f7b1effd640 1 -- 172.21.15.40:0/3366500681 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121759158 0 0) 0x7f7b200b7b30 con 0x7f7b2013e2c0 2023-07-29T13:28:12.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.630+0000 7f7b1effd640 10 monclient(hunting): _negotiate 2023-07-29T13:28:12.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.630+0000 7f7b1effd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:12.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.630+0000 7f7b1effd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:12.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.630+0000 7f7b1effd640 10 monclient(hunting): my global_id is 4270 2023-07-29T13:28:12.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.630+0000 7f7b1effd640 1 -- 172.21.15.40:0/3366500681 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b08003520 con 0x7f7b2013e2c0 2023-07-29T13:28:12.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.631+0000 7f7b24d61640 10 monclient(hunting): handle_auth_done global_id 4275 payload 274 2023-07-29T13:28:12.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.631+0000 7f7b24d61640 1 -- 172.21.15.40:0/3366500681 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7b200b7760 msgr2=0x7f7b2013dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:12.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.631+0000 7f7b24d61640 1 --2- 172.21.15.40:0/3366500681 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7b200b7760 0x7f7b2013dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:12.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.631+0000 7f7b24d61640 1 -- 172.21.15.40:0/3366500681 >> v1:172.21.15.45:6789/0 conn(0x7f7b2013e2c0 legacy=0x7f7b20144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:12.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.631+0000 7f7b24d61640 10 monclient: _finish_hunting 0 2023-07-29T13:28:12.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.631+0000 7f7b24d61640 1 monclient: found mon.noname-a 2023-07-29T13:28:12.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.631+0000 7f7b24d61640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:12.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.631+0000 7f7b24d61640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:12.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.631+0000 7f7b24d61640 1 -- 172.21.15.40:0/3366500681 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b200a9210 con 0x7f7b20144a40 2023-07-29T13:28:12.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.633+0000 7f7b24d61640 1 --2- 172.21.15.40:0/3366500681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b20144a40 0x7f7b2014b420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009d90 tx=0x7f7b1802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49be447dae0d5704 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:12.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 1 -- 172.21.15.40:0/3366500681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1802cc50 con 0x7f7b20144a40 2023-07-29T13:28:12.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:12.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:12.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 10 monclient: dump: 2023-07-29T13:28:12.639 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:12.640 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:12.640 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:12.640 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:12.640 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:12.640 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:12.640 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:12.640 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:12.640 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:12.641 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:12.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 10 monclient: _finish_auth 0 2023-07-29T13:28:12.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 10 monclient: _check_auth_tickets 2023-07-29T13:28:12.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:12.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 1 -- 172.21.15.40:0/3366500681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1802cdb0 con 0x7f7b20144a40 2023-07-29T13:28:12.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b267c4640 5 monclient: authenticate success, global_id 4275 2023-07-29T13:28:12.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b267c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:12.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:12.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b267c4640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:12.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b267c4640 4 set_mon_vals no callback set 2023-07-29T13:28:12.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 1 -- 172.21.15.40:0/3366500681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18034660 con 0x7f7b20144a40 2023-07-29T13:28:12.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:12.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:12.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.637+0000 7f7b1effd640 10 monclient: dump: 2023-07-29T13:28:12.642 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:12.642 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:12.643 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:12.643 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:12.643 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:12.643 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:12.643 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:12.643 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:12.643 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:12.643 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:12.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.638+0000 7f7b267c4640 10 monclient: shutdown 2023-07-29T13:28:12.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.638+0000 7f7b267c4640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:12.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.638+0000 7f7b267c4640 1 -- 172.21.15.40:0/3366500681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b20144a40 msgr2=0x7f7b2014b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:12.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.638+0000 7f7b267c4640 1 --2- 172.21.15.40:0/3366500681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b20144a40 0x7f7b2014b420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009d90 tx=0x7f7b1802bea0 comp rx=0 tx=0).stop 2023-07-29T13:28:12.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.638+0000 7f7b267c4640 1 -- 172.21.15.40:0/3366500681 shutdown_connections 2023-07-29T13:28:12.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.638+0000 7f7b267c4640 1 --2- 172.21.15.40:0/3366500681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b20144a40 0x7f7b2014b420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:12.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.638+0000 7f7b267c4640 1 --2- 172.21.15.40:0/3366500681 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7b200b7760 0x7f7b2013dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:12.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.638+0000 7f7b267c4640 1 -- 172.21.15.40:0/3366500681 >> 172.21.15.40:0/3366500681 conn(0x7f7b200713e0 msgr2=0x7f7b200717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:12.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.638+0000 7f7b267c4640 1 -- 172.21.15.40:0/3366500681 shutdown_connections 2023-07-29T13:28:12.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.638+0000 7f7b267c4640 1 -- 172.21.15.40:0/3366500681 wait complete. 2023-07-29T13:28:12.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.639+0000 7f7b267c4640 10 monclient: build_initial_monmap 2023-07-29T13:28:12.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.639+0000 7f7b267c4640 10 monclient: monmap: 2023-07-29T13:28:12.647 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:12.647 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:12.648 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:12.648 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:12.648 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:12.648 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:12.648 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:12.648 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:12.648 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:12.648 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.639+0000 7f7b267c4640 1 librados: starting msgr at 2023-07-29T13:28:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.639+0000 7f7b267c4640 1 librados: starting objecter 2023-07-29T13:28:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.639+0000 7f7b267c4640 1 Processor -- start 2023-07-29T13:28:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.639+0000 7f7b267c4640 1 -- start start 2023-07-29T13:28:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.639+0000 7f7b267c4640 1 librados: setting wanted keys 2023-07-29T13:28:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.639+0000 7f7b267c4640 1 librados: calling monclient init 2023-07-29T13:28:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.639+0000 7f7b267c4640 10 monclient: init 2023-07-29T13:28:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T13:28:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b200b7760 0x7f7b200b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 10 monclient(hunting): picked mon.noname-a con 0x7f7b200b7760 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7b2013e2c0 0x7f7b200b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 10 monclient(hunting): picked mon.noname-c con 0x7f7b2013e2c0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b20144a40 0x7f7b20139740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 10 monclient(hunting): picked mon.noname-b con 0x7f7b20144a40 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b200b61c0 con 0x7f7b200b7760 2023-07-29T13:28:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b200b5e40 con 0x7f7b20144a40 2023-07-29T13:28:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b200b5fb0 con 0x7f7b2013e2c0 2023-07-29T13:28:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b267c4640 10 monclient(hunting): authenticate will time out at 1414.773605s 2023-07-29T13:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.640+0000 7f7b1ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b200b7760 0x7f7b200b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.641+0000 7f7b24d61640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b20144a40 0x7f7b20139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.641+0000 7f7b1ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b200b7760 0x7f7b200b66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60428/0 (socket says 172.21.15.40:60428) 2023-07-29T13:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.641+0000 7f7b1ffff640 1 -- 172.21.15.40:0/2487493159 learned_addr learned my addr 172.21.15.40:0/2487493159 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.641+0000 7f7b1ffff640 10 monclient(hunting): get_auth_request con 0x7f7b200b7760 auth_method 0 2023-07-29T13:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.641+0000 7f7b1ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.641+0000 7f7b1ffff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.641+0000 7f7b1ffff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.641+0000 7f7b24d61640 10 monclient(hunting): get_auth_request con 0x7f7b20144a40 auth_method 0 2023-07-29T13:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.641+0000 7f7b24d61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.641+0000 7f7b24d61640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.641+0000 7f7b24d61640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.642+0000 7f7b1ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.642+0000 7f7b1ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.642+0000 7f7b1ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.642+0000 7f7b24d61640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.642+0000 7f7b24d61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.642+0000 7f7b24d61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.645+0000 7f7b1ffff640 10 monclient(hunting): handle_auth_done global_id 4278 payload 824 2023-07-29T13:28:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.645+0000 7f7b1ffff640 1 -- 172.21.15.40:0/2487493159 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7b2013e2c0 msgr2=0x7f7b200b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.645+0000 7f7b1ffff640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7b2013e2c0 0x7f7b200b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.645+0000 7f7b1ffff640 1 -- 172.21.15.40:0/2487493159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b20144a40 msgr2=0x7f7b20139740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.645+0000 7f7b1ffff640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b20144a40 0x7f7b20139740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.645+0000 7f7b1ffff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.645+0000 7f7b1ffff640 1 monclient: found mon.noname-a 2023-07-29T13:28:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.645+0000 7f7b1ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.645+0000 7f7b1ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.645+0000 7f7b1ffff640 1 -- 172.21.15.40:0/2487493159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b18009a70 con 0x7f7b200b7760 2023-07-29T13:28:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.648+0000 7f7b1ffff640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b200b7760 0x7f7b200b66e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000b4a0 tx=0x7f7b1000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.648+0000 7f7b1cff9640 10 client.?.objecter ms_handle_connect 0x7f7b200b7760 2023-07-29T13:28:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.648+0000 7f7b1cff9640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 1 -- 172.21.15.40:0/2487493159 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10010610 con 0x7f7b200b7760 2023-07-29T13:28:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 10 monclient: dump: 2023-07-29T13:28:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:12.658 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:12.658 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:12.658 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:12.658 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:12.658 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:12.658 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:12.658 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:12.659 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 10 monclient: _finish_auth 0 2023-07-29T13:28:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 10 monclient: _check_auth_tickets 2023-07-29T13:28:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 1 -- 172.21.15.40:0/2487493159 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b10010770 con 0x7f7b200b7760 2023-07-29T13:28:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 1 -- 172.21.15.40:0/2487493159 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10010b20 con 0x7f7b200b7760 2023-07-29T13:28:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1cff9640 10 monclient: dump: 2023-07-29T13:28:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:12.661 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b1e7fc640 4 set_mon_vals no callback set 2023-07-29T13:28:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b267c4640 5 monclient: authenticate success, global_id 4278 2023-07-29T13:28:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b267c4640 10 monclient: _renew_subs 2023-07-29T13:28:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b267c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b267c4640 1 -- 172.21.15.40:0/2487493159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b200b4b60 con 0x7f7b200b7760 2023-07-29T13:28:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b267c4640 10 client.4278.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b267c4640 10 monclient: _renew_subs 2023-07-29T13:28:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b267c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b267c4640 1 -- 172.21.15.40:0/2487493159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b200b03e0 con 0x7f7b200b7760 2023-07-29T13:28:12.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.653+0000 7f7b267c4640 1 librados: init done 2023-07-29T13:28:12.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.654+0000 7f7b267c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:12.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.654+0000 7f7b267c4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:12.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.654+0000 7f7b267c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:12.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.654+0000 7f7b267c4640 1 -- 172.21.15.40:0/2487493159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b200678e0 con 0x7f7b200b7760 2023-07-29T13:28:12.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.676+0000 7f7b1cff9640 1 -- 172.21.15.40:0/2487493159 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 6) v1 ==== 88153+0+0 (secure 0 0 0) 0x7f7b10010c80 con 0x7f7b200b7760 2023-07-29T13:28:12.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.677+0000 7f7b1cff9640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7b0c06bf20 0x7f7b0c06e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:12.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.677+0000 7f7b1f7fe640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7b0c06bf20 0x7f7b0c06e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:12.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.677+0000 7f7b1f7fe640 10 monclient: get_auth_request con 0x7f7b0c06bf20 auth_method 0 2023-07-29T13:28:12.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.677+0000 7f7b1cff9640 10 client.4278.objecter ms_handle_connect 0x7f7b0c06bf20 2023-07-29T13:28:12.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.677+0000 7f7b1f7fe640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7b0c06bf20 0x7f7b0c06e3d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b20143b80 tx=0x7f7b14007930 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:12.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.952+0000 7f7b1cff9640 1 -- 172.21.15.40:0/2487493159 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f7b1005bae0 con 0x7f7b200b7760 2023-07-29T13:28:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.952+0000 7f7b1cff9640 10 client.4278.objecter ms_dispatch 0x7f7b20000b90 osd_map(21..21 src has 1..21) v4 2023-07-29T13:28:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.952+0000 7f7b1cff9640 3 client.4278.objecter handle_osd_map got epochs [21,21] > 0 2023-07-29T13:28:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.952+0000 7f7b1cff9640 3 client.4278.objecter handle_osd_map decoding full epoch 21 2023-07-29T13:28:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.953+0000 7f7b1cff9640 20 client.4278.objecter dump_active .. 0 homeless 2023-07-29T13:28:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.953+0000 7f7b1cff9640 1 -- 172.21.15.40:0/2487493159 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f7b10053f10 con 0x7f7b200b7760 2023-07-29T13:28:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.953+0000 7f7b1cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:12.953+0000 7f7b1cff9640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:13.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.156+0000 7f7b267c4640 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2023-07-29T13:28:13.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.156+0000 7f7b267c4640 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2023-07-29T13:28:13.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.156+0000 7f7b267c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:13.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.156+0000 7f7b267c4640 1 -- 172.21.15.40:0/2487493159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7b2013acd0 con 0x7f7b200b7760 2023-07-29T13:28:13.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.769+0000 7f7b1cff9640 1 -- 172.21.15.40:0/2487493159 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f7b100542e0 con 0x7f7b200b7760 2023-07-29T13:28:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.784+0000 7f7b1cff9640 1 -- 172.21.15.40:0/2487493159 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v22) v1 ==== 173+0+0 (secure 0 0 0) 0x7f7b10066e10 con 0x7f7b200b7760 2023-07-29T13:28:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.784+0000 7f7b1cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2023-07-29T13:28:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.784+0000 7f7b1cff9640 10 monclient: _finish_command 2 = system:0 enabled application 'rbd' on pool 'rbd' 2023-07-29T13:28:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:enabled application 'rbd' on pool 'rbd' 2023-07-29T13:28:13.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 10 librados: watch_flush enter 2023-07-29T13:28:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 10 librados: watch_flush exit 2023-07-29T13:28:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 20 client.4278.objecter shutdown clearing up homeless session... 2023-07-29T13:28:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 10 client.4278.objecter successfully canceled tick 2023-07-29T13:28:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 1 -- 172.21.15.40:0/2487493159 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7b0c06bf20 msgr2=0x7f7b0c06e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7b0c06bf20 0x7f7b0c06e3d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b20143b80 tx=0x7f7b14007930 comp rx=0 tx=0).stop 2023-07-29T13:28:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 10 monclient: shutdown 2023-07-29T13:28:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 1 -- 172.21.15.40:0/2487493159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b200b7760 msgr2=0x7f7b200b66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b200b7760 0x7f7b200b66e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000b4a0 tx=0x7f7b1000b960 comp rx=0 tx=0).stop 2023-07-29T13:28:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 1 -- 172.21.15.40:0/2487493159 shutdown_connections 2023-07-29T13:28:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7b0c06bf20 0x7f7b0c06e3d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b20144a40 0x7f7b20139740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7b2013e2c0 0x7f7b200b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.786+0000 7f7b267c4640 1 --2- 172.21.15.40:0/2487493159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b200b7760 0x7f7b200b66e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:13.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.787+0000 7f7b267c4640 1 -- 172.21.15.40:0/2487493159 >> 172.21.15.40:0/2487493159 conn(0x7f7b200713e0 msgr2=0x7f7b20068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:13.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.787+0000 7f7b267c4640 1 -- 172.21.15.40:0/2487493159 shutdown_connections 2023-07-29T13:28:13.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.787+0000 7f7b267c4640 1 -- 172.21.15.40:0/2487493159 wait complete. 2023-07-29T13:28:13.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.787+0000 7f7b267c4640 1 librados: shutdown 2023-07-29T13:28:13.801 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-07-29T13:28:13.802 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-29T13:28:13.802 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-07-29T13:28:13.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-07-29T13:28:13.803 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-07-29T13:28:13.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.910+0000 7f0974944640 10 monclient: get_monmap_and_config 2023-07-29T13:28:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.910+0000 7f0974944640 10 monclient: build_initial_monmap 2023-07-29T13:28:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.910+0000 7f0974944640 10 monclient: monmap: 2023-07-29T13:28:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:13.912142+0000 2023-07-29T13:28:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:13.912142+0000 2023-07-29T13:28:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:13.914 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.911+0000 7f0974944640 1 Processor -- start 2023-07-29T13:28:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.911+0000 7f0974944640 1 -- start start 2023-07-29T13:28:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.911+0000 7f0974944640 10 monclient: init 2023-07-29T13:28:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2023-07-29T13:28:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09700b3ae0 0x7f097013dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 10 monclient(hunting): picked mon.noname-c con 0x7f09700b3ae0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f09700a8910 0x7f097013e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 10 monclient(hunting): picked mon.noname-b con 0x7f09700a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0970140c10 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09700b3eb0 con 0x7f0970140c10 2023-07-29T13:28:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09700b0f10 con 0x7f09700b3ae0 2023-07-29T13:28:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09700a9950 con 0x7f09700a8910 2023-07-29T13:28:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.912+0000 7f0974944640 10 monclient(hunting): authenticate will time out at 1416.045512s 2023-07-29T13:28:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.913+0000 7f096ed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0970140c10 0x7f0970141070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51218/0 (socket says 172.21.15.40:51218) 2023-07-29T13:28:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.913+0000 7f096ed9d640 1 -- 172.21.15.40:0/1733582864 learned_addr learned my addr 172.21.15.40:0/1733582864 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:28:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.915+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1033340219 0 0) 0x7f09700b3eb0 con 0x7f0970140c10 2023-07-29T13:28:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.915+0000 7f096d59a640 10 monclient(hunting): _negotiate 2023-07-29T13:28:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.915+0000 7f096d59a640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.915+0000 7f096d59a640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.915+0000 7f096d59a640 10 monclient(hunting): my global_id is 4281 2023-07-29T13:28:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.915+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0958003520 con 0x7f0970140c10 2023-07-29T13:28:13.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2363518474 0 0) 0x7f0958003520 con 0x7f0970140c10 2023-07-29T13:28:13.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f09700a8910 msgr2=0x7f097013e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:13.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 --2- 172.21.15.40:0/1733582864 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f09700a8910 0x7f097013e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:13.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 >> v2:172.21.15.45:3300/0 conn(0x7f09700b3ae0 msgr2=0x7f097013dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:13.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 --2- 172.21.15.40:0/1733582864 >> v2:172.21.15.45:3300/0 conn(0x7f09700b3ae0 0x7f097013dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:13.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient: _finish_hunting 0 2023-07-29T13:28:13.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 monclient: found mon.noname-a-legacy 2023-07-29T13:28:13.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:13.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.40:6789/0 2023-07-29T13:28:13.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700a9c30 con 0x7f0970140c10 2023-07-29T13:28:13.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient: _finish_auth 0 2023-07-29T13:28:13.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient: _check_auth_tickets 2023-07-29T13:28:13.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:13.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3319386150 0 0) 0x7f09680019f0 con 0x7f0970140c10 2023-07-29T13:28:13.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:13.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-07-29T13:28:13.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient: dump: 2023-07-29T13:28:13.923 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:13.923 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:13.923 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:13.924 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:13.924 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:13.924 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:13.924 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:13.924 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:13.924 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:13.924 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:13.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] but i'm connected to v1:172.21.15.40:6789/0, reconnecting 2023-07-29T13:28:13.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:13.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 >> v1:172.21.15.40:6789/0 conn(0x7f0970140c10 legacy=0x7f0970141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:13.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T13:28:13.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 --2- 172.21.15.40:0/1733582864 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f09700a8910 0x7f0958003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:13.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient(hunting): picked mon.c con 0x7f09700a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:13.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 --2- 172.21.15.40:0/1733582864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09580040d0 0x7f09580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:13.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient(hunting): picked mon.a con 0x7f09580040d0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:13.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 --2- 172.21.15.40:0/1733582864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0958004e20 0x7f0958005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:13.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient(hunting): picked mon.b con 0x7f0958004e20 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:13.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:13.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09700a9950 con 0x7f09580040d0 2023-07-29T13:28:13.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:13.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095800bc10 con 0x7f0958004e20 2023-07-29T13:28:13.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:13.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095800bdd0 con 0x7f09700a8910 2023-07-29T13:28:13.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:13.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.919+0000 7f096d59a640 10 monclient(hunting): _finish_auth 0 2023-07-29T13:28:13.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.921+0000 7f096dd9b640 1 --2- 172.21.15.40:0/1733582864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09580040d0 0x7f09580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:13.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.923+0000 7f096dd9b640 10 monclient(hunting): get_auth_request con 0x7f09580040d0 auth_method 0 2023-07-29T13:28:13.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.923+0000 7f096dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:13.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.923+0000 7f096dd9b640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:13.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.923+0000 7f096dd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2023-07-29T13:28:13.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.924+0000 7f096dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:13.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.924+0000 7f096dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:13.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.924+0000 7f096dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-07-29T13:28:13.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.934+0000 7f096dd9b640 10 monclient(hunting): handle_auth_done global_id 4281 payload 293 2023-07-29T13:28:13.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.934+0000 7f096dd9b640 1 -- 172.21.15.40:0/1733582864 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f09700a8910 msgr2=0x7f0958003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:13.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.934+0000 7f096dd9b640 1 --2- 172.21.15.40:0/1733582864 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f09700a8910 0x7f0958003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:13.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f096dd9b640 1 -- 172.21.15.40:0/1733582864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0958004e20 msgr2=0x7f0958005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:13.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f096dd9b640 1 --2- 172.21.15.40:0/1733582864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0958004e20 0x7f0958005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:13.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f096dd9b640 10 monclient: _finish_hunting 0 2023-07-29T13:28:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f096dd9b640 1 monclient: found mon.a 2023-07-29T13:28:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f096dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f096dd9b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f096dd9b640 1 -- 172.21.15.40:0/1733582864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0958003520 con 0x7f09580040d0 2023-07-29T13:28:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f096dd9b640 10 monclient: _finish_auth 0 2023-07-29T13:28:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f096dd9b640 10 monclient: _check_auth_tickets 2023-07-29T13:28:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f096dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f0974944640 5 monclient: authenticate success, global_id 4281 2023-07-29T13:28:13.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.935+0000 7f0974944640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:13.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.937+0000 7f096dd9b640 1 --2- 172.21.15.40:0/1733582864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09580040d0 0x7f09580047c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0960009860 tx=0x7f09580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd14d8fed31cff9 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:13.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960032600 con 0x7f09580040d0 2023-07-29T13:28:13.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f096d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:13.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f096d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:13.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f096d59a640 10 monclient: dump: 2023-07-29T13:28:13.944 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:13.944 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:13.944 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:13.944 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:13.944 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:13.944 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:13.944 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:13.945 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:13.945 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:13.945 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:13.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096002e050 con 0x7f09580040d0 2023-07-29T13:28:13.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f0974944640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:13.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f096d59a640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:13.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f0974944640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:13.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f096d59a640 1 -- 172.21.15.40:0/1733582864 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09600329c0 con 0x7f09580040d0 2023-07-29T13:28:13.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f0974944640 4 set_mon_vals no callback set 2023-07-29T13:28:13.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f096d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:13.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f096d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:13.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.941+0000 7f096d59a640 10 monclient: dump: 2023-07-29T13:28:13.946 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:13.946 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:13.946 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:13.946 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:13.947 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:13.947 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:13.947 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:13.947 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:13.947 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:13.947 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:13.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.942+0000 7f0974944640 10 monclient: shutdown 2023-07-29T13:28:13.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.942+0000 7f0974944640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:13.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.942+0000 7f0974944640 1 -- 172.21.15.40:0/1733582864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09580040d0 msgr2=0x7f09580047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:13.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.942+0000 7f0974944640 1 --2- 172.21.15.40:0/1733582864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09580040d0 0x7f09580047c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0960009860 tx=0x7f09580057f0 comp rx=0 tx=0).stop 2023-07-29T13:28:13.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.942+0000 7f096ed9d640 1 -- 172.21.15.40:0/1733582864 reap_dead start 2023-07-29T13:28:13.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.942+0000 7f0974944640 1 -- 172.21.15.40:0/1733582864 shutdown_connections 2023-07-29T13:28:13.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.942+0000 7f0974944640 1 -- 172.21.15.40:0/1733582864 >> 172.21.15.40:0/1733582864 conn(0x7f0970070fc0 msgr2=0x7f09700713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:13.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.942+0000 7f0974944640 1 -- 172.21.15.40:0/1733582864 shutdown_connections 2023-07-29T13:28:13.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.942+0000 7f0974944640 1 -- 172.21.15.40:0/1733582864 wait complete. 2023-07-29T13:28:13.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.943+0000 7f0974944640 10 monclient: build_initial_monmap 2023-07-29T13:28:13.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.943+0000 7f0974944640 10 monclient: monmap: 2023-07-29T13:28:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:13.952 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 1 librados: starting msgr at 2023-07-29T13:28:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 1 librados: starting objecter 2023-07-29T13:28:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 1 Processor -- start 2023-07-29T13:28:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 1 -- start start 2023-07-29T13:28:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 1 librados: setting wanted keys 2023-07-29T13:28:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 1 librados: calling monclient init 2023-07-29T13:28:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 10 monclient: init 2023-07-29T13:28:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T13:28:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09580040d0 0x7f097013fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 10 monclient(hunting): picked mon.noname-b con 0x7f09580040d0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.944+0000 7f0974944640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0970143ca0 0x7f09701d5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f0974944640 10 monclient(hunting): picked mon.noname-c con 0x7f0970143ca0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f0974944640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0970144650 0x7f09701d5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f0974944640 10 monclient(hunting): picked mon.noname-a con 0x7f0970144650 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f0974944640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f0974944640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970144180 con 0x7f0970144650 2023-07-29T13:28:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f0974944640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f0974944640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09701d3340 con 0x7f09580040d0 2023-07-29T13:28:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f0974944640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f0974944640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0970140680 con 0x7f0970143ca0 2023-07-29T13:28:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f0974944640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f0974944640 10 monclient(hunting): authenticate will time out at 1416.078216s 2023-07-29T13:28:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.945+0000 7f096ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0970144650 0x7f09701d5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.946+0000 7f096ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0970144650 0x7f09701d5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60440/0 (socket says 172.21.15.40:60440) 2023-07-29T13:28:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.946+0000 7f096ed9d640 1 -- 172.21.15.40:0/2133090844 learned_addr learned my addr 172.21.15.40:0/2133090844 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.946+0000 7f096ed9d640 10 monclient(hunting): get_auth_request con 0x7f0970144650 auth_method 0 2023-07-29T13:28:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.946+0000 7f096ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.946+0000 7f096ed9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.946+0000 7f096ed9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.946+0000 7f096ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.946+0000 7f096ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.946+0000 7f096ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:13.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.949+0000 7f096ed9d640 10 monclient(hunting): handle_auth_done global_id 4284 payload 824 2023-07-29T13:28:13.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.949+0000 7f096ed9d640 1 -- 172.21.15.40:0/2133090844 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0970143ca0 msgr2=0x7f09701d5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:13.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.949+0000 7f096ed9d640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0970143ca0 0x7f09701d5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:13.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.949+0000 7f096ed9d640 1 -- 172.21.15.40:0/2133090844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09580040d0 msgr2=0x7f097013fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:13.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.949+0000 7f096ed9d640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09580040d0 0x7f097013fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:13.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.949+0000 7f096ed9d640 10 monclient: _finish_hunting 0 2023-07-29T13:28:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.949+0000 7f096ed9d640 1 monclient: found mon.noname-a 2023-07-29T13:28:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.949+0000 7f096ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.949+0000 7f096ed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.949+0000 7f096ed9d640 1 -- 172.21.15.40:0/2133090844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0970143450 con 0x7f0970144650 2023-07-29T13:28:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.952+0000 7f096ed9d640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0970144650 0x7f09701d5cd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0970066900 tx=0x7f096800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.952+0000 7f095effd640 10 client.?.objecter ms_handle_connect 0x7f0970144650 2023-07-29T13:28:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.952+0000 7f095effd640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 1 -- 172.21.15.40:0/2133090844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968004d60 con 0x7f0970144650 2023-07-29T13:28:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 10 monclient: dump: 2023-07-29T13:28:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:13.961 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 10 monclient: _finish_auth 0 2023-07-29T13:28:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 10 monclient: _check_auth_tickets 2023-07-29T13:28:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 1 -- 172.21.15.40:0/2133090844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0968007070 con 0x7f0970144650 2023-07-29T13:28:13.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f0974944640 5 monclient: authenticate success, global_id 4284 2023-07-29T13:28:13.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f0974944640 10 monclient: _renew_subs 2023-07-29T13:28:13.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f0974944640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:13.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f0974944640 1 -- 172.21.15.40:0/2133090844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0970136350 con 0x7f0970144650 2023-07-29T13:28:13.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:13.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f096cd99640 4 set_mon_vals no callback set 2023-07-29T13:28:13.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f0974944640 10 client.4284.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 1 -- 172.21.15.40:0/2133090844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968007420 con 0x7f0970144650 2023-07-29T13:28:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f095effd640 10 monclient: dump: 2023-07-29T13:28:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:13.964 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:13.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f0974944640 10 monclient: _renew_subs 2023-07-29T13:28:13.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f0974944640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:13.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.957+0000 7f0974944640 1 -- 172.21.15.40:0/2133090844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09701367e0 con 0x7f0970144650 2023-07-29T13:28:13.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.958+0000 7f0974944640 1 librados: init done 2023-07-29T13:28:13.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.959+0000 7f0974944640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:13.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.959+0000 7f0974944640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:13.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.959+0000 7f0974944640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:13.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.959+0000 7f0974944640 1 -- 172.21.15.40:0/2133090844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09701442f0 con 0x7f0970144650 2023-07-29T13:28:13.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.993+0000 7f095effd640 1 -- 172.21.15.40:0/2133090844 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f0968007580 con 0x7f0970144650 2023-07-29T13:28:13.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.993+0000 7f095effd640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f095006bf60 0x7f095006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:13.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.994+0000 7f096e59c640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f095006bf60 0x7f095006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:13.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.994+0000 7f096e59c640 10 monclient: get_auth_request con 0x7f095006bf60 auth_method 0 2023-07-29T13:28:13.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.994+0000 7f095effd640 10 client.4284.objecter ms_handle_connect 0x7f095006bf60 2023-07-29T13:28:13.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:13.994+0000 7f096e59c640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f095006bf60 0x7f095006e410 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0964009710 tx=0x7f0964006d00 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:14.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.035+0000 7f095effd640 1 -- 172.21.15.40:0/2133090844 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f096801b2f0 con 0x7f0970144650 2023-07-29T13:28:14.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.036+0000 7f095effd640 10 client.4284.objecter ms_dispatch 0x7f0970000b90 osd_map(22..22 src has 1..22) v4 2023-07-29T13:28:14.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.036+0000 7f095effd640 3 client.4284.objecter handle_osd_map got epochs [22,22] > 0 2023-07-29T13:28:14.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.036+0000 7f095effd640 3 client.4284.objecter handle_osd_map decoding full epoch 22 2023-07-29T13:28:14.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.036+0000 7f095effd640 20 client.4284.objecter dump_active .. 0 homeless 2023-07-29T13:28:14.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.324+0000 7f095effd640 1 -- 172.21.15.40:0/2133090844 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f0968053ae0 con 0x7f0970144650 2023-07-29T13:28:14.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.324+0000 7f095effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:14.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.324+0000 7f095effd640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:14.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.602+0000 7f0974944640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2023-07-29T13:28:14.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.602+0000 7f0974944640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2023-07-29T13:28:14.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.602+0000 7f0974944640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:14.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.602+0000 7f0974944640 1 -- 172.21.15.40:0/2133090844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f09700a8570 con 0x7f0970144650 2023-07-29T13:28:14.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.786+0000 7f095effd640 1 -- 172.21.15.40:0/2133090844 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v7) v1 ==== 74+0+169778 (secure 0 0 0) 0x7f0968057100 con 0x7f0970144650 2023-07-29T13:28:14.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.786+0000 7f095effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2023-07-29T13:28:14.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.786+0000 7f095effd640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:14.790 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:28:14.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 10 librados: watch_flush enter 2023-07-29T13:28:14.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 10 librados: watch_flush exit 2023-07-29T13:28:14.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 20 client.4284.objecter shutdown clearing up homeless session... 2023-07-29T13:28:14.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 10 client.4284.objecter successfully canceled tick 2023-07-29T13:28:14.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 -- 172.21.15.40:0/2133090844 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f095006bf60 msgr2=0x7f095006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:14.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f095006bf60 0x7f095006e410 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0964009710 tx=0x7f0964006d00 comp rx=0 tx=0).stop 2023-07-29T13:28:14.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 10 monclient: shutdown 2023-07-29T13:28:14.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:14.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 -- 172.21.15.40:0/2133090844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0970144650 msgr2=0x7f09701d5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:14.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0970144650 0x7f09701d5cd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0970066900 tx=0x7f096800cb30 comp rx=0 tx=0).stop 2023-07-29T13:28:14.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 -- 172.21.15.40:0/2133090844 shutdown_connections 2023-07-29T13:28:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f095006bf60 0x7f095006e410 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0970144650 0x7f09701d5cd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0970143ca0 0x7f09701d5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 --2- 172.21.15.40:0/2133090844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09580040d0 0x7f097013fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 -- 172.21.15.40:0/2133090844 >> 172.21.15.40:0/2133090844 conn(0x7f0970070fc0 msgr2=0x7f09700b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 -- 172.21.15.40:0/2133090844 shutdown_connections 2023-07-29T13:28:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 -- 172.21.15.40:0/2133090844 wait complete. 2023-07-29T13:28:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.789+0000 7f0974944640 1 librados: shutdown 2023-07-29T13:28:14.803 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"active_gid":4152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":84611},{"type":"v1","addr":"172.21.15.45:6801","nonce":84611}]},"active_addr":"172.21.15.45:6801/84611","active_change":"2023-07-29T13:27:28.655294+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":11,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":470065371}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2356897719}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2007758460}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":989027757}]}]} 2023-07-29T13:28:14.814 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-07-29T13:28:14.814 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-07-29T13:28:14.815 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-29T13:28:14.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.958+0000 7f76dca06640 10 monclient: get_monmap_and_config 2023-07-29T13:28:14.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.958+0000 7f76dca06640 10 monclient: build_initial_monmap 2023-07-29T13:28:14.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.958+0000 7f76dca06640 10 monclient: monmap: 2023-07-29T13:28:14.960 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:14.959510+0000 2023-07-29T13:28:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:14.959510+0000 2023-07-29T13:28:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:14.962 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 1 Processor -- start 2023-07-29T13:28:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 1 -- start start 2023-07-29T13:28:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 10 monclient: init 2023-07-29T13:28:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2023-07-29T13:28:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f76d80b3ae0 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76d813e3a0 0x7f76d8140c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 10 monclient(hunting): picked mon.noname-a con 0x7f76d813e3a0 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f76d80a8910 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d80b3eb0 con 0x7f76d80b3ae0 2023-07-29T13:28:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d80a9950 con 0x7f76d80a8910 2023-07-29T13:28:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d80b0f10 con 0x7f76d813e3a0 2023-07-29T13:28:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.959+0000 7f76dca06640 10 monclient(hunting): authenticate will time out at 1417.093008s 2023-07-29T13:28:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.961+0000 7f76cffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76d813e3a0 0x7f76d8140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:14.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.962+0000 7f76cffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76d813e3a0 0x7f76d8140c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60456/0 (socket says 172.21.15.40:60456) 2023-07-29T13:28:14.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.962+0000 7f76cffff640 1 -- 172.21.15.40:0/3338616331 learned_addr learned my addr 172.21.15.40:0/3338616331 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:14.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.962+0000 7f76cffff640 10 monclient(hunting): get_auth_request con 0x7f76d813e3a0 auth_method 0 2023-07-29T13:28:14.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.962+0000 7f76cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:14.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.962+0000 7f76cffff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:14.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.962+0000 7f76cffff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:14.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.963+0000 7f76cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:14.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.963+0000 7f76cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:14.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.963+0000 7f76cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:14.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.982+0000 7f76cffff640 10 monclient(hunting): handle_auth_done global_id 4287 payload 274 2023-07-29T13:28:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.982+0000 7f76cffff640 1 -- 172.21.15.40:0/3338616331 >> v1:172.21.15.45:6789/0 conn(0x7f76d80a8910 legacy=0x7f76d814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.982+0000 7f76cffff640 1 -- 172.21.15.40:0/3338616331 >> v1:172.21.15.40:6789/0 conn(0x7f76d80b3ae0 legacy=0x7f76d813dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.982+0000 7f76cffff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.982+0000 7f76cffff640 1 monclient: found mon.noname-a 2023-07-29T13:28:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.982+0000 7f76cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.982+0000 7f76cffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.982+0000 7f76cffff640 1 -- 172.21.15.40:0/3338616331 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d80a9c30 con 0x7f76d813e3a0 2023-07-29T13:28:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:14.991+0000 7f76cffff640 1 --2- 172.21.15.40:0/3338616331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76d813e3a0 0x7f76d8140c10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76d80668d0 tx=0x7f76c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1fee0dc40f1838 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:15.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 1 -- 172.21.15.40:0/3338616331 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0031640 con 0x7f76d813e3a0 2023-07-29T13:28:15.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:15.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:15.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 10 monclient: dump: 2023-07-29T13:28:15.013 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:15.013 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:15.013 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.013 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.013 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:15.013 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:15.014 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:15.014 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:15.014 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:15.014 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:15.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 10 monclient: _finish_auth 0 2023-07-29T13:28:15.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 10 monclient: _check_auth_tickets 2023-07-29T13:28:15.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:15.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 1 -- 172.21.15.40:0/3338616331 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c00317a0 con 0x7f76d813e3a0 2023-07-29T13:28:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 1 -- 172.21.15.40:0/3338616331 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0031b30 con 0x7f76d813e3a0 2023-07-29T13:28:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76d5d9b640 10 monclient: dump: 2023-07-29T13:28:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:15.016 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76dca06640 5 monclient: authenticate success, global_id 4287 2023-07-29T13:28:15.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76dca06640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:15.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.010+0000 7f76dca06640 4 set_mon_vals no callback set 2023-07-29T13:28:15.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.011+0000 7f76dca06640 10 monclient: shutdown 2023-07-29T13:28:15.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.011+0000 7f76dca06640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:15.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.011+0000 7f76dca06640 1 -- 172.21.15.40:0/3338616331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76d813e3a0 msgr2=0x7f76d8140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:15.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.011+0000 7f76dca06640 1 --2- 172.21.15.40:0/3338616331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76d813e3a0 0x7f76d8140c10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76d80668d0 tx=0x7f76c002b8f0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.011+0000 7f76dca06640 1 -- 172.21.15.40:0/3338616331 shutdown_connections 2023-07-29T13:28:15.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.011+0000 7f76dca06640 1 --2- 172.21.15.40:0/3338616331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76d813e3a0 0x7f76d8140c10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.011+0000 7f76dca06640 1 -- 172.21.15.40:0/3338616331 >> 172.21.15.40:0/3338616331 conn(0x7f76d8070fc0 msgr2=0x7f76d80713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:15.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.011+0000 7f76dca06640 1 -- 172.21.15.40:0/3338616331 shutdown_connections 2023-07-29T13:28:15.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.011+0000 7f76dca06640 1 -- 172.21.15.40:0/3338616331 wait complete. 2023-07-29T13:28:15.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.012+0000 7f76dca06640 10 monclient: build_initial_monmap 2023-07-29T13:28:15.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.012+0000 7f76dca06640 10 monclient: monmap: 2023-07-29T13:28:15.020 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:15.020 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:15.020 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:15.020 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:15.020 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:15.021 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.013+0000 7f76dca06640 1 librados: starting msgr at 2023-07-29T13:28:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.013+0000 7f76dca06640 1 librados: starting objecter 2023-07-29T13:28:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.013+0000 7f76dca06640 1 Processor -- start 2023-07-29T13:28:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.013+0000 7f76dca06640 1 -- start start 2023-07-29T13:28:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.013+0000 7f76dca06640 1 librados: setting wanted keys 2023-07-29T13:28:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.013+0000 7f76dca06640 1 librados: calling monclient init 2023-07-29T13:28:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.013+0000 7f76dca06640 10 monclient: init 2023-07-29T13:28:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T13:28:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76d80a8910 0x7f76d81d8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 10 monclient(hunting): picked mon.noname-a con 0x7f76d80a8910 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:15.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d80b3ae0 0x7f76d80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:15.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 10 monclient(hunting): picked mon.noname-b con 0x7f76d80b3ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:15.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f76d813e3a0 0x7f76d80ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:15.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 10 monclient(hunting): picked mon.noname-c con 0x7f76d813e3a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:15.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:15.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80ada00 con 0x7f76d80a8910 2023-07-29T13:28:15.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:15.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d81d91a0 con 0x7f76d80b3ae0 2023-07-29T13:28:15.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:15.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80afe90 con 0x7f76d813e3a0 2023-07-29T13:28:15.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:15.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.014+0000 7f76dca06640 10 monclient(hunting): authenticate will time out at 1417.147302s 2023-07-29T13:28:15.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.015+0000 7f76cffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d80b3ae0 0x7f76d80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:15.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.016+0000 7f76cffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d80b3ae0 0x7f76d80af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:42358/0 (socket says 172.21.15.40:42358) 2023-07-29T13:28:15.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.016+0000 7f76cffff640 1 -- 172.21.15.40:0/2489354802 learned_addr learned my addr 172.21.15.40:0/2489354802 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:15.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.016+0000 7f76cffff640 10 monclient(hunting): get_auth_request con 0x7f76d80b3ae0 auth_method 0 2023-07-29T13:28:15.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.016+0000 7f76cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:15.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.016+0000 7f76cffff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:15.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.016+0000 7f76cffff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:15.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.017+0000 7f76d6d9d640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f76d813e3a0 0x7f76d80ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:15.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.018+0000 7f76d6d9d640 10 monclient(hunting): get_auth_request con 0x7f76d813e3a0 auth_method 0 2023-07-29T13:28:15.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.018+0000 7f76d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:15.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.018+0000 7f76d6d9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:15.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.018+0000 7f76d6d9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:15.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.019+0000 7f76d6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:15.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.019+0000 7f76d6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:15.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.019+0000 7f76d6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:15.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.023+0000 7f76d6d9d640 10 monclient(hunting): handle_auth_done global_id 4211 payload 824 2023-07-29T13:28:15.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.023+0000 7f76d6d9d640 1 -- 172.21.15.40:0/2489354802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d80b3ae0 msgr2=0x7f76d80af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:15.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.023+0000 7f76d6d9d640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d80b3ae0 0x7f76d80af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.023+0000 7f76d6d9d640 1 -- 172.21.15.40:0/2489354802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76d80a8910 msgr2=0x7f76d81d8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:15.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.023+0000 7f76d6d9d640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76d80a8910 0x7f76d81d8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.023+0000 7f76d6d9d640 10 monclient: _finish_hunting 0 2023-07-29T13:28:15.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.023+0000 7f76d6d9d640 1 monclient: found mon.noname-c 2023-07-29T13:28:15.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.023+0000 7f76d6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:15.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.023+0000 7f76d6d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:28:15.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.023+0000 7f76d6d9d640 1 -- 172.21.15.40:0/2489354802 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c00094d0 con 0x7f76d813e3a0 2023-07-29T13:28:15.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.026+0000 7f76d6d9d640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f76d813e3a0 0x7f76d80ade50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f76d000c6c0 tx=0x7f76d000cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:15.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.026+0000 7f76ceffd640 10 client.?.objecter ms_handle_connect 0x7f76d813e3a0 2023-07-29T13:28:15.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.026+0000 7f76ceffd640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:15.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 1 -- 172.21.15.40:0/2489354802 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0003b70 con 0x7f76d813e3a0 2023-07-29T13:28:15.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:15.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:15.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 10 monclient: dump: 2023-07-29T13:28:15.087 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:15.087 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:15.087 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.087 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.088 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:15.088 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:15.088 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:15.088 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:15.088 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:15.088 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:15.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 10 monclient: _finish_auth 0 2023-07-29T13:28:15.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 10 monclient: _check_auth_tickets 2023-07-29T13:28:15.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:15.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 1 -- 172.21.15.40:0/2489354802 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d0003cd0 con 0x7f76d813e3a0 2023-07-29T13:28:15.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76dca06640 5 monclient: authenticate success, global_id 4211 2023-07-29T13:28:15.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76dca06640 10 monclient: _renew_subs 2023-07-29T13:28:15.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76dca06640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:15.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76dca06640 1 -- 172.21.15.40:0/2489354802 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d80ac290 con 0x7f76d813e3a0 2023-07-29T13:28:15.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 1 -- 172.21.15.40:0/2489354802 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d00052f0 con 0x7f76d813e3a0 2023-07-29T13:28:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76ceffd640 10 monclient: dump: 2023-07-29T13:28:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:15.091 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76d559a640 4 set_mon_vals no callback set 2023-07-29T13:28:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76dca06640 10 client.4211.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:15.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76dca06640 10 monclient: _renew_subs 2023-07-29T13:28:15.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76dca06640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:15.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76dca06640 1 -- 172.21.15.40:0/2489354802 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d813ac60 con 0x7f76d813e3a0 2023-07-29T13:28:15.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.084+0000 7f76dca06640 1 librados: init done 2023-07-29T13:28:15.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.085+0000 7f76dca06640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:15.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.085+0000 7f76dca06640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:15.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.085+0000 7f76dca06640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:15.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.085+0000 7f76dca06640 1 -- 172.21.15.40:0/2489354802 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d80adb70 con 0x7f76d813e3a0 2023-07-29T13:28:15.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.373+0000 7f76ceffd640 1 -- 172.21.15.40:0/2489354802 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f76d0005450 con 0x7f76d813e3a0 2023-07-29T13:28:15.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.373+0000 7f76ceffd640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f76b806bfb0 0x7f76b806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:15.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.374+0000 7f76d659c640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f76b806bfb0 0x7f76b806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.374+0000 7f76d659c640 10 monclient: get_auth_request con 0x7f76b806bfb0 auth_method 0 2023-07-29T13:28:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.374+0000 7f76d659c640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f76b806bfb0 0x7f76b806e460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76c8001560 tx=0x7f76c8007990 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.374+0000 7f76ceffd640 10 client.4211.objecter ms_handle_connect 0x7f76b806bfb0 2023-07-29T13:28:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.388+0000 7f76ceffd640 1 -- 172.21.15.40:0/2489354802 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f76d005c200 con 0x7f76d813e3a0 2023-07-29T13:28:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.388+0000 7f76ceffd640 10 client.4211.objecter ms_dispatch 0x7f76d8000b90 osd_map(23..23 src has 1..23) v4 2023-07-29T13:28:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.388+0000 7f76ceffd640 3 client.4211.objecter handle_osd_map got epochs [23,23] > 0 2023-07-29T13:28:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.388+0000 7f76ceffd640 3 client.4211.objecter handle_osd_map decoding full epoch 23 2023-07-29T13:28:15.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.388+0000 7f76ceffd640 20 client.4211.objecter dump_active .. 0 homeless 2023-07-29T13:28:15.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.388+0000 7f76ceffd640 1 -- 172.21.15.40:0/2489354802 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f76d0054580 con 0x7f76d813e3a0 2023-07-29T13:28:15.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.388+0000 7f76ceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:15.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.388+0000 7f76ceffd640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:15.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.566+0000 7f76dca06640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:15.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.566+0000 7f76dca06640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:15.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.566+0000 7f76dca06640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:15.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.566+0000 7f76dca06640 1 -- 172.21.15.40:0/2489354802 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76d80adb70 con 0x7f76d813e3a0 2023-07-29T13:28:15.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.671+0000 7f76ceffd640 1 -- 172.21.15.40:0/2489354802 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+14159 (secure 0 0 0) 0x7f76d80adb70 con 0x7f76d813e3a0 2023-07-29T13:28:15.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.671+0000 7f76ceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:15.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.671+0000 7f76ceffd640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:15.674 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:28:15.674 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","created":"2023-07-29T13:27:17.185104+0000","modified":"2023-07-29T13:28:14.763630+0000","last_up_change":"2023-07-29T13:28:06.434639+0000","last_in_change":"2023-07-29T13:27:22.831496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-29T13:28:05.611243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-29T13:28:11.546076+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6069a944-26b4-4b00-94e2-f91c485e8957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":85570},{"type":"v1","addr":"172.21.15.40:6809","nonce":85570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":85570},{"type":"v1","addr":"172.21.15.40:6811","nonce":85570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":85570},{"type":"v1","addr":"172.21.15.40:6815","nonce":85570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":85570},{"type":"v1","addr":"172.21.15.40:6813","nonce":85570}]},"public_addr":"172.21.15.40:6809/85570","cluster_addr":"172.21.15.40:6811/85570","heartbeat_back_addr":"172.21.15.40:6815/85570","heartbeat_front_addr":"172.21.15.40:6813/85570","state":["exists","up"]},{"osd":1,"uuid":"ab18edd4-e1cb-4815-b5a8-af2b4b2e790b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":85573},{"type":"v1","addr":"172.21.15.40:6817","nonce":85573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":85573},{"type":"v1","addr":"172.21.15.40:6819","nonce":85573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":85573},{"type":"v1","addr":"172.21.15.40:6823","nonce":85573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":85573},{"type":"v1","addr":"172.21.15.40:6821","nonce":85573}]},"public_addr":"172.21.15.40:6817/85573","cluster_addr":"172.21.15.40:6819/85573","heartbeat_back_addr":"172.21.15.40:6823/85573","heartbeat_front_addr":"172.21.15.40:6821/85573","state":["exists","up"]},{"osd":2,"uuid":"4575bc2c-4d43-46d3-95ba-1dc2c94278e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":85572},{"type":"v1","addr":"172.21.15.40:6801","nonce":85572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":85572},{"type":"v1","addr":"172.21.15.40:6803","nonce":85572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":85572},{"type":"v1","addr":"172.21.15.40:6807","nonce":85572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":85572},{"type":"v1","addr":"172.21.15.40:6805","nonce":85572}]},"public_addr":"172.21.15.40:6801/85572","cluster_addr":"172.21.15.40:6803/85572","heartbeat_back_addr":"172.21.15.40:6807/85572","heartbeat_front_addr":"172.21.15.40:6805/85572","state":["exists","up"]},{"osd":3,"uuid":"e7224640-7d33-4bb7-81a5-99fc26da7763","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":85571},{"type":"v1","addr":"172.21.15.40:6825","nonce":85571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":85571},{"type":"v1","addr":"172.21.15.40:6827","nonce":85571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":85571},{"type":"v1","addr":"172.21.15.40:6831","nonce":85571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":85571},{"type":"v1","addr":"172.21.15.40:6829","nonce":85571}]},"public_addr":"172.21.15.40:6825/85571","cluster_addr":"172.21.15.40:6827/85571","heartbeat_back_addr":"172.21.15.40:6831/85571","heartbeat_front_addr":"172.21.15.40:6829/85571","state":["exists","up"]},{"osd":4,"uuid":"23cd76f6-788b-48cf-9438-e931659d3a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":85183},{"type":"v1","addr":"172.21.15.45:6827","nonce":85183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":85183},{"type":"v1","addr":"172.21.15.45:6829","nonce":85183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":85183},{"type":"v1","addr":"172.21.15.45:6833","nonce":85183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":85183},{"type":"v1","addr":"172.21.15.45:6831","nonce":85183}]},"public_addr":"172.21.15.45:6827/85183","cluster_addr":"172.21.15.45:6829/85183","heartbeat_back_addr":"172.21.15.45:6833/85183","heartbeat_front_addr":"172.21.15.45:6831/85183","state":["exists","up"]},{"osd":5,"uuid":"0ddb1e3f-0f65-40d6-ab50-5dc521774445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":85181},{"type":"v1","addr":"172.21.15.45:6813","nonce":85181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":85181},{"type":"v1","addr":"172.21.15.45:6818","nonce":85181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6823","nonce":85181},{"type":"v1","addr":"172.21.15.45:6825","nonce":85181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":85181},{"type":"v1","addr":"172.21.15.45:6821","nonce":85181}]},"public_addr":"172.21.15.45:6813/85181","cluster_addr":"172.21.15.45:6818/85181","heartbeat_back_addr":"172.21.15.45:6825/85181","heartbeat_front_addr":"172.21.15.45:6821/85181","state":["exists","up"]},{"osd":6,"uuid":"4b9d2830-9f2c-438c-9c0b-b6692e21f921","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":85184},{"type":"v1","addr":"172.21.15.45:6811","nonce":85184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":85184},{"type":"v1","addr":"172.21.15.45:6815","nonce":85184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":85184},{"type":"v1","addr":"172.21.15.45:6824","nonce":85184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":85184},{"type":"v1","addr":"172.21.15.45:6819","nonce":85184}]},"public_addr":"172.21.15.45:6811/85184","cluster_addr":"172.21.15.45:6815/85184","heartbeat_back_addr":"172.21.15.45:6824/85184","heartbeat_front_addr":"172.21.15.45:6819/85184","state":["exists","up"]},{"osd":7,"uuid":"6593807d-34a1-431b-aa23-24e8340d887b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":85182},{"type":"v1","addr":"172.21.15.45:6803","nonce":85182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":85182},{"type":"v1","addr":"172.21.15.45:6805","nonce":85182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":85182},{"type":"v1","addr":"172.21.15.45:6809","nonce":85182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":85182},{"type":"v1","addr":"172.21.15.45:6807","nonce":85182}]},"public_addr":"172.21.15.45:6803/85182","cluster_addr":"172.21.15.45:6805/85182","heartbeat_back_addr":"172.21.15.45:6809/85182","heartbeat_front_addr":"172.21.15.45:6807/85182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.729512+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.651149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.683349+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:03.275000+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:03.102406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:01.537435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:01.740924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:00.468651+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-29T13:28:15.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 10 librados: watch_flush enter 2023-07-29T13:28:15.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 10 librados: watch_flush exit 2023-07-29T13:28:15.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 20 client.4211.objecter shutdown clearing up homeless session... 2023-07-29T13:28:15.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 10 client.4211.objecter successfully canceled tick 2023-07-29T13:28:15.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 -- 172.21.15.40:0/2489354802 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f76b806bfb0 msgr2=0x7f76b806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:15.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f76b806bfb0 0x7f76b806e460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76c8001560 tx=0x7f76c8007990 comp rx=0 tx=0).stop 2023-07-29T13:28:15.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 10 monclient: shutdown 2023-07-29T13:28:15.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:15.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 -- 172.21.15.40:0/2489354802 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f76d813e3a0 msgr2=0x7f76d80ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:15.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f76d813e3a0 0x7f76d80ade50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f76d000c6c0 tx=0x7f76d000cb80 comp rx=0 tx=0).stop 2023-07-29T13:28:15.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 -- 172.21.15.40:0/2489354802 shutdown_connections 2023-07-29T13:28:15.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f76b806bfb0 0x7f76b806e460 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d80b3ae0 0x7f76d80af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f76d813e3a0 0x7f76d80ade50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 --2- 172.21.15.40:0/2489354802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76d80a8910 0x7f76d81d8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 -- 172.21.15.40:0/2489354802 >> 172.21.15.40:0/2489354802 conn(0x7f76d8070fc0 msgr2=0x7f76d80b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:15.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 -- 172.21.15.40:0/2489354802 shutdown_connections 2023-07-29T13:28:15.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 -- 172.21.15.40:0/2489354802 wait complete. 2023-07-29T13:28:15.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.674+0000 7f76dca06640 1 librados: shutdown 2023-07-29T13:28:15.687 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-07-29T13:28:15.687 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-29T13:28:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.791+0000 7fbd2f1f5640 10 monclient: get_monmap_and_config 2023-07-29T13:28:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.791+0000 7fbd2f1f5640 10 monclient: build_initial_monmap 2023-07-29T13:28:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.791+0000 7fbd2f1f5640 10 monclient: monmap: 2023-07-29T13:28:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:15.792667+0000 2023-07-29T13:28:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:15.792667+0000 2023-07-29T13:28:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:15.795 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 1 Processor -- start 2023-07-29T13:28:15.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 1 -- start start 2023-07-29T13:28:15.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 10 monclient: init 2023-07-29T13:28:15.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:15.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 10 monclient: _add_conns ranks=[3,2,1,4,0] 2023-07-29T13:28:15.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbd280b3ae0 0x7fbd2813dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:15.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 10 monclient(hunting): picked mon.noname-a con 0x7fbd280b3ae0 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:15.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd280a8910 0x7fbd2813e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:15.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 10 monclient(hunting): picked mon.noname-b con 0x7fbd280a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:15.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbd28140c10 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:15.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd280b3eb0 con 0x7fbd28140c10 2023-07-29T13:28:15.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:15.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd280b0f10 con 0x7fbd280b3ae0 2023-07-29T13:28:15.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd280a9950 con 0x7fbd280a8910 2023-07-29T13:28:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.792+0000 7fbd2f1f5640 10 monclient(hunting): authenticate will time out at 1417.925948s 2023-07-29T13:28:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.793+0000 7fbd27fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd280a8910 0x7fbd2813e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.793+0000 7fbd2cf91640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbd280b3ae0 0x7fbd2813dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.793+0000 7fbd2d792640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbd28140c10 0x7fbd28141070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.40:46244/0 (socket says 172.21.15.40:46244) 2023-07-29T13:28:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.793+0000 7fbd2d792640 1 -- 172.21.15.40:0/214618586 learned_addr learned my addr 172.21.15.40:0/214618586 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:28:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd27fff640 10 monclient(hunting): get_auth_request con 0x7fbd280a8910 auth_method 0 2023-07-29T13:28:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd27fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd27fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd2cf91640 10 monclient(hunting): get_auth_request con 0x7fbd280b3ae0 auth_method 0 2023-07-29T13:28:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd2cf91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd2cf91640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd2cf91640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd2cf91640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd2cf91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.794+0000 7fbd2cf91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:15.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.796+0000 7fbd27fff640 10 monclient(hunting): handle_auth_done global_id 4214 payload 274 2023-07-29T13:28:15.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.796+0000 7fbd27fff640 1 -- 172.21.15.40:0/214618586 >> v2:172.21.15.40:3300/0 conn(0x7fbd280b3ae0 msgr2=0x7fbd2813dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:15.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.796+0000 7fbd27fff640 1 --2- 172.21.15.40:0/214618586 >> v2:172.21.15.40:3300/0 conn(0x7fbd280b3ae0 0x7fbd2813dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.796+0000 7fbd27fff640 1 -- 172.21.15.40:0/214618586 >> v1:172.21.15.45:6789/0 conn(0x7fbd28140c10 legacy=0x7fbd28141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.796+0000 7fbd27fff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.796+0000 7fbd27fff640 1 monclient: found mon.noname-b 2023-07-29T13:28:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.796+0000 7fbd27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.796+0000 7fbd27fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.796+0000 7fbd277fe640 1 -- 172.21.15.40:0/214618586 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513859938 0 0) 0x7fbd280b3eb0 con 0x7fbd28140c10 2023-07-29T13:28:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.796+0000 7fbd27fff640 1 -- 172.21.15.40:0/214618586 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd280a9c30 con 0x7fbd280a8910 2023-07-29T13:28:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.796+0000 7fbd277fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2023-07-29T13:28:15.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.798+0000 7fbd27fff640 1 --2- 172.21.15.40:0/214618586 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd280a8910 0x7fbd2813e1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbd140097a0 tx=0x7fbd1402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fdf2e866b53af51 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 1 -- 172.21.15.40:0/214618586 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1402d040 con 0x7fbd280a8910 2023-07-29T13:28:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 10 monclient: dump: 2023-07-29T13:28:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:15.807 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 10 monclient: _finish_auth 0 2023-07-29T13:28:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 10 monclient: _check_auth_tickets 2023-07-29T13:28:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 1 -- 172.21.15.40:0/214618586 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd14031810 con 0x7fbd280a8910 2023-07-29T13:28:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 5 monclient: authenticate success, global_id 4214 2023-07-29T13:28:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 1 -- 172.21.15.40:0/214618586 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14031ba0 con 0x7fbd280a8910 2023-07-29T13:28:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 4 set_mon_vals no callback set 2023-07-29T13:28:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd277fe640 10 monclient: dump: 2023-07-29T13:28:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.810 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:15.810 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:15.810 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:15.810 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:15.810 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:15.810 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:15.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 10 monclient: shutdown 2023-07-29T13:28:15.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/214618586 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd280a8910 msgr2=0x7fbd2813e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 1 --2- 172.21.15.40:0/214618586 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd280a8910 0x7fbd2813e1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbd140097a0 tx=0x7fbd1402b8b0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/214618586 shutdown_connections 2023-07-29T13:28:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 1 --2- 172.21.15.40:0/214618586 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd280a8910 0x7fbd2813e1d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 1 --2- 172.21.15.40:0/214618586 >> v2:172.21.15.40:3300/0 conn(0x7fbd280b3ae0 0x7fbd2813dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.802+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/214618586 >> 172.21.15.40:0/214618586 conn(0x7fbd28070fc0 msgr2=0x7fbd280713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.803+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/214618586 shutdown_connections 2023-07-29T13:28:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.803+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/214618586 wait complete. 2023-07-29T13:28:15.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.803+0000 7fbd2f1f5640 10 monclient: build_initial_monmap 2023-07-29T13:28:15.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.803+0000 7fbd2f1f5640 10 monclient: monmap: 2023-07-29T13:28:15.812 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:15.812 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:15.812 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:15.812 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:15.812 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:15.813 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.803+0000 7fbd2f1f5640 1 librados: starting msgr at 2023-07-29T13:28:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 1 librados: starting objecter 2023-07-29T13:28:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 1 Processor -- start 2023-07-29T13:28:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 1 -- start start 2023-07-29T13:28:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 1 librados: setting wanted keys 2023-07-29T13:28:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 1 librados: calling monclient init 2023-07-29T13:28:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 10 monclient: init 2023-07-29T13:28:15.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:15.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T13:28:15.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd280a8910 0x7fbd280b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:15.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 10 monclient(hunting): picked mon.noname-a con 0x7fbd280a8910 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd280b3ae0 0x7fbd281d9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 10 monclient(hunting): picked mon.noname-b con 0x7fbd280b3ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd28140c10 0x7fbd281d9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 10 monclient(hunting): picked mon.noname-c con 0x7fbd28140c10 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd280aa9b0 con 0x7fbd280a8910 2023-07-29T13:28:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd28144540 con 0x7fbd280b3ae0 2023-07-29T13:28:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd280b7930 con 0x7fbd28140c10 2023-07-29T13:28:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.804+0000 7fbd2f1f5640 10 monclient(hunting): authenticate will time out at 1417.938025s 2023-07-29T13:28:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.805+0000 7fbd2d792640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd28140c10 0x7fbd281d9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.805+0000 7fbd27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd280b3ae0 0x7fbd281d9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.805+0000 7fbd2cf91640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd280a8910 0x7fbd280b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd27fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd280b3ae0 0x7fbd281d9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:42370/0 (socket says 172.21.15.40:42370) 2023-07-29T13:28:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd27fff640 1 -- 172.21.15.40:0/49659597 learned_addr learned my addr 172.21.15.40:0/49659597 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd27fff640 10 monclient(hunting): get_auth_request con 0x7fbd280b3ae0 auth_method 0 2023-07-29T13:28:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2d792640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd28140c10 0x7fbd281d9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:48946/0 (socket says 172.21.15.40:48946) 2023-07-29T13:28:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd27fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd27fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2d792640 10 monclient(hunting): get_auth_request con 0x7fbd28140c10 auth_method 0 2023-07-29T13:28:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2d792640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2d792640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2d792640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2cf91640 10 monclient(hunting): get_auth_request con 0x7fbd280a8910 auth_method 0 2023-07-29T13:28:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2cf91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2cf91640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:15.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2cf91640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:15.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:15.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:15.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:15.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2d792640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:15.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2d792640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:15.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2d792640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:15.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2cf91640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:15.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2cf91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:15.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.806+0000 7fbd2cf91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:15.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.809+0000 7fbd27fff640 10 monclient(hunting): handle_auth_done global_id 4282 payload 824 2023-07-29T13:28:15.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.809+0000 7fbd27fff640 1 -- 172.21.15.40:0/49659597 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd28140c10 msgr2=0x7fbd281d9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:15.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.809+0000 7fbd27fff640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd28140c10 0x7fbd281d9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.809+0000 7fbd27fff640 1 -- 172.21.15.40:0/49659597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd280a8910 msgr2=0x7fbd280b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:15.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.809+0000 7fbd27fff640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd280a8910 0x7fbd280b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:15.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.809+0000 7fbd27fff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.809+0000 7fbd27fff640 1 monclient: found mon.noname-b 2023-07-29T13:28:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.809+0000 7fbd27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.809+0000 7fbd27fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.809+0000 7fbd27fff640 1 -- 172.21.15.40:0/49659597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd14009400 con 0x7fbd280b3ae0 2023-07-29T13:28:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.812+0000 7fbd27fff640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd280b3ae0 0x7fbd281d9220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd140098c0 tx=0x7fbd140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.812+0000 7fbd257fa640 10 client.?.objecter ms_handle_connect 0x7fbd280b3ae0 2023-07-29T13:28:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.812+0000 7fbd257fa640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 1 -- 172.21.15.40:0/49659597 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14006e10 con 0x7fbd280b3ae0 2023-07-29T13:28:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 10 monclient: dump: 2023-07-29T13:28:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.826 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:15.826 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:15.826 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:15.826 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:15.826 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:15.826 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:15.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 10 monclient: _finish_auth 0 2023-07-29T13:28:15.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 10 monclient: _check_auth_tickets 2023-07-29T13:28:15.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:15.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 1 -- 172.21.15.40:0/49659597 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1403ad60 con 0x7fbd280b3ae0 2023-07-29T13:28:15.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:15.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 1 -- 172.21.15.40:0/49659597 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14044620 con 0x7fbd280b3ae0 2023-07-29T13:28:15.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd2f1f5640 5 monclient: authenticate success, global_id 4282 2023-07-29T13:28:15.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:15.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:15.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd257fa640 10 monclient: dump: 2023-07-29T13:28:15.828 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:15.828 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:15.828 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.828 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:15.828 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:15.828 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:15.828 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:15.829 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:15.829 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:15.829 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:15.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd2f1f5640 10 monclient: _renew_subs 2023-07-29T13:28:15.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd26ffd640 4 set_mon_vals no callback set 2023-07-29T13:28:15.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd2f1f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:15.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.815+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/49659597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd28144a60 con 0x7fbd280b3ae0 2023-07-29T13:28:15.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.816+0000 7fbd2f1f5640 10 client.4282.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:15.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.816+0000 7fbd2f1f5640 10 monclient: _renew_subs 2023-07-29T13:28:15.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.816+0000 7fbd2f1f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:15.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.816+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/49659597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd28144020 con 0x7fbd280b3ae0 2023-07-29T13:28:15.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.816+0000 7fbd2f1f5640 1 librados: init done 2023-07-29T13:28:15.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.816+0000 7fbd2f1f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:15.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.816+0000 7fbd2f1f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:15.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.816+0000 7fbd2f1f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:15.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:15.816+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/49659597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd280b1600 con 0x7fbd280b3ae0 2023-07-29T13:28:16.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.073+0000 7fbd257fa640 1 -- 172.21.15.40:0/49659597 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fbd1404d440 con 0x7fbd280b3ae0 2023-07-29T13:28:16.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.073+0000 7fbd257fa640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbd0c06bfb0 0x7fbd0c06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.074+0000 7fbd2cf91640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbd0c06bfb0 0x7fbd0c06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.074+0000 7fbd2cf91640 10 monclient: get_auth_request con 0x7fbd0c06bfb0 auth_method 0 2023-07-29T13:28:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.074+0000 7fbd257fa640 10 client.4282.objecter ms_handle_connect 0x7fbd0c06bfb0 2023-07-29T13:28:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.074+0000 7fbd2cf91640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbd0c06bfb0 0x7fbd0c06e460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009b40 tx=0x7fbd1c009190 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.078+0000 7fbd257fa640 1 -- 172.21.15.40:0/49659597 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbd1407e1b0 con 0x7fbd280b3ae0 2023-07-29T13:28:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.078+0000 7fbd257fa640 10 client.4282.objecter ms_dispatch 0x7fbd28000b90 osd_map(23..23 src has 1..23) v4 2023-07-29T13:28:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.078+0000 7fbd257fa640 3 client.4282.objecter handle_osd_map got epochs [23,23] > 0 2023-07-29T13:28:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.078+0000 7fbd257fa640 3 client.4282.objecter handle_osd_map decoding full epoch 23 2023-07-29T13:28:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.078+0000 7fbd257fa640 20 client.4282.objecter dump_active .. 0 homeless 2023-07-29T13:28:16.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.089+0000 7fbd257fa640 1 -- 172.21.15.40:0/49659597 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7fbd140765e0 con 0x7fbd280b3ae0 2023-07-29T13:28:16.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.089+0000 7fbd257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:16.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.089+0000 7fbd257fa640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:16.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.340+0000 7fbd2f1f5640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:16.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.340+0000 7fbd2f1f5640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:16.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.340+0000 7fbd2f1f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:16.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.340+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/49659597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd280b54d0 con 0x7fbd280b3ae0 2023-07-29T13:28:16.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.375+0000 7fbd257fa640 1 -- 172.21.15.40:0/49659597 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+14146 (secure 0 0 0) 0x7fbd14035090 con 0x7fbd280b3ae0 2023-07-29T13:28:16.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.375+0000 7fbd257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T13:28:16.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.375+0000 7fbd257fa640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:16.377 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:28:16.378 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","created":"2023-07-29T13:27:17.185104+0000","modified":"2023-07-29T13:28:15.852587+0000","last_up_change":"2023-07-29T13:28:06.434639+0000","last_in_change":"2023-07-29T13:27:22.831496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-29T13:28:05.611243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-29T13:28:11.546076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6069a944-26b4-4b00-94e2-f91c485e8957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":85570},{"type":"v1","addr":"172.21.15.40:6809","nonce":85570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":85570},{"type":"v1","addr":"172.21.15.40:6811","nonce":85570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":85570},{"type":"v1","addr":"172.21.15.40:6815","nonce":85570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":85570},{"type":"v1","addr":"172.21.15.40:6813","nonce":85570}]},"public_addr":"172.21.15.40:6809/85570","cluster_addr":"172.21.15.40:6811/85570","heartbeat_back_addr":"172.21.15.40:6815/85570","heartbeat_front_addr":"172.21.15.40:6813/85570","state":["exists","up"]},{"osd":1,"uuid":"ab18edd4-e1cb-4815-b5a8-af2b4b2e790b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":85573},{"type":"v1","addr":"172.21.15.40:6817","nonce":85573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":85573},{"type":"v1","addr":"172.21.15.40:6819","nonce":85573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":85573},{"type":"v1","addr":"172.21.15.40:6823","nonce":85573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":85573},{"type":"v1","addr":"172.21.15.40:6821","nonce":85573}]},"public_addr":"172.21.15.40:6817/85573","cluster_addr":"172.21.15.40:6819/85573","heartbeat_back_addr":"172.21.15.40:6823/85573","heartbeat_front_addr":"172.21.15.40:6821/85573","state":["exists","up"]},{"osd":2,"uuid":"4575bc2c-4d43-46d3-95ba-1dc2c94278e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":85572},{"type":"v1","addr":"172.21.15.40:6801","nonce":85572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":85572},{"type":"v1","addr":"172.21.15.40:6803","nonce":85572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":85572},{"type":"v1","addr":"172.21.15.40:6807","nonce":85572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":85572},{"type":"v1","addr":"172.21.15.40:6805","nonce":85572}]},"public_addr":"172.21.15.40:6801/85572","cluster_addr":"172.21.15.40:6803/85572","heartbeat_back_addr":"172.21.15.40:6807/85572","heartbeat_front_addr":"172.21.15.40:6805/85572","state":["exists","up"]},{"osd":3,"uuid":"e7224640-7d33-4bb7-81a5-99fc26da7763","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":85571},{"type":"v1","addr":"172.21.15.40:6825","nonce":85571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":85571},{"type":"v1","addr":"172.21.15.40:6827","nonce":85571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":85571},{"type":"v1","addr":"172.21.15.40:6831","nonce":85571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":85571},{"type":"v1","addr":"172.21.15.40:6829","nonce":85571}]},"public_addr":"172.21.15.40:6825/85571","cluster_addr":"172.21.15.40:6827/85571","heartbeat_back_addr":"172.21.15.40:6831/85571","heartbeat_front_addr":"172.21.15.40:6829/85571","state":["exists","up"]},{"osd":4,"uuid":"23cd76f6-788b-48cf-9438-e931659d3a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":85183},{"type":"v1","addr":"172.21.15.45:6827","nonce":85183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":85183},{"type":"v1","addr":"172.21.15.45:6829","nonce":85183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":85183},{"type":"v1","addr":"172.21.15.45:6833","nonce":85183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":85183},{"type":"v1","addr":"172.21.15.45:6831","nonce":85183}]},"public_addr":"172.21.15.45:6827/85183","cluster_addr":"172.21.15.45:6829/85183","heartbeat_back_addr":"172.21.15.45:6833/85183","heartbeat_front_addr":"172.21.15.45:6831/85183","state":["exists","up"]},{"osd":5,"uuid":"0ddb1e3f-0f65-40d6-ab50-5dc521774445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":85181},{"type":"v1","addr":"172.21.15.45:6813","nonce":85181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":85181},{"type":"v1","addr":"172.21.15.45:6818","nonce":85181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6823","nonce":85181},{"type":"v1","addr":"172.21.15.45:6825","nonce":85181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":85181},{"type":"v1","addr":"172.21.15.45:6821","nonce":85181}]},"public_addr":"172.21.15.45:6813/85181","cluster_addr":"172.21.15.45:6818/85181","heartbeat_back_addr":"172.21.15.45:6825/85181","heartbeat_front_addr":"172.21.15.45:6821/85181","state":["exists","up"]},{"osd":6,"uuid":"4b9d2830-9f2c-438c-9c0b-b6692e21f921","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":85184},{"type":"v1","addr":"172.21.15.45:6811","nonce":85184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":85184},{"type":"v1","addr":"172.21.15.45:6815","nonce":85184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":85184},{"type":"v1","addr":"172.21.15.45:6824","nonce":85184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":85184},{"type":"v1","addr":"172.21.15.45:6819","nonce":85184}]},"public_addr":"172.21.15.45:6811/85184","cluster_addr":"172.21.15.45:6815/85184","heartbeat_back_addr":"172.21.15.45:6824/85184","heartbeat_front_addr":"172.21.15.45:6819/85184","state":["exists","up"]},{"osd":7,"uuid":"6593807d-34a1-431b-aa23-24e8340d887b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":85182},{"type":"v1","addr":"172.21.15.45:6803","nonce":85182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":85182},{"type":"v1","addr":"172.21.15.45:6805","nonce":85182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":85182},{"type":"v1","addr":"172.21.15.45:6809","nonce":85182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":85182},{"type":"v1","addr":"172.21.15.45:6807","nonce":85182}]},"public_addr":"172.21.15.45:6803/85182","cluster_addr":"172.21.15.45:6805/85182","heartbeat_back_addr":"172.21.15.45:6809/85182","heartbeat_front_addr":"172.21.15.45:6807/85182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.729512+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.651149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.683349+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:03.275000+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:03.102406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:01.537435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:01.740924+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:00.468651+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-29T13:28:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.376+0000 7fbd2f1f5640 10 librados: watch_flush enter 2023-07-29T13:28:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 10 librados: watch_flush exit 2023-07-29T13:28:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 20 client.4282.objecter shutdown clearing up homeless session... 2023-07-29T13:28:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 10 client.4282.objecter successfully canceled tick 2023-07-29T13:28:16.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/49659597 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbd0c06bfb0 msgr2=0x7fbd0c06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbd0c06bfb0 0x7fbd0c06e460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009b40 tx=0x7fbd1c009190 comp rx=0 tx=0).stop 2023-07-29T13:28:16.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 10 monclient: shutdown 2023-07-29T13:28:16.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/49659597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd280b3ae0 msgr2=0x7fbd281d9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd280b3ae0 0x7fbd281d9220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd140098c0 tx=0x7fbd140046c0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/49659597 shutdown_connections 2023-07-29T13:28:16.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbd0c06bfb0 0x7fbd0c06e460 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd280b3ae0 0x7fbd281d9220 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbd28140c10 0x7fbd281d9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 --2- 172.21.15.40:0/49659597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd280a8910 0x7fbd280b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/49659597 >> 172.21.15.40:0/49659597 conn(0x7fbd28070fc0 msgr2=0x7fbd28141070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/49659597 shutdown_connections 2023-07-29T13:28:16.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 -- 172.21.15.40:0/49659597 wait complete. 2023-07-29T13:28:16.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.377+0000 7fbd2f1f5640 1 librados: shutdown 2023-07-29T13:28:16.390 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-07-29T13:28:16.390 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-07-29T13:28:16.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-07-29T13:28:16.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-07-29T13:28:16.392 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-07-29T13:28:16.392 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-07-29T13:28:16.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-07-29T13:28:16.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-07-29T13:28:16.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.575+0000 7f8874a4f640 10 monclient: get_monmap_and_config 2023-07-29T13:28:16.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.575+0000 7f8874a4f640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.575+0000 7f8874a4f640 10 monclient: monmap: 2023-07-29T13:28:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:16.576387+0000 2023-07-29T13:28:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:16.576387+0000 2023-07-29T13:28:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:16.580 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.576+0000 7f8874a4f640 1 Processor -- start 2023-07-29T13:28:16.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.576+0000 7f05a9803640 10 monclient: get_monmap_and_config 2023-07-29T13:28:16.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f05a9803640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f05a9803640 10 monclient: monmap: 2023-07-29T13:28:16.582 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.582 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.582 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:16.578562+0000 2023-07-29T13:28:16.582 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:16.578562+0000 2023-07-29T13:28:16.582 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.582 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.582 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:16.583 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:16.583 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:16.583 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:16.583 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:16.583 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.576+0000 7f8874a4f640 1 -- start start 2023-07-29T13:28:16.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.576+0000 7f8874a4f640 10 monclient: init 2023-07-29T13:28:16.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2023-07-29T13:28:16.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f88700afce0 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8870140230 0x7f8870140620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 10 monclient(hunting): picked mon.noname-b con 0x7f8870140230 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8870140b60 0x7f8870147560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 10 monclient(hunting): picked mon.noname-a con 0x7f8870140b60 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88700ae960 con 0x7f88700afce0 2023-07-29T13:28:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7fe30b05a640 10 monclient: get_monmap_and_config 2023-07-29T13:28:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7fe30b05a640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7fe30b05a640 10 monclient: monmap: 2023-07-29T13:28:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:16.578735+0000 2023-07-29T13:28:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:16.578735+0000 2023-07-29T13:28:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:16.587 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887006a070 con 0x7f8870140b60 2023-07-29T13:28:16.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88700ae720 con 0x7f8870140230 2023-07-29T13:28:16.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.577+0000 7f8874a4f640 10 monclient(hunting): authenticate will time out at 1418.710858s 2023-07-29T13:28:16.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7f0883cbc640 10 monclient: get_monmap_and_config 2023-07-29T13:28:16.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7f0883cbc640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7f0883cbc640 10 monclient: monmap: 2023-07-29T13:28:16.589 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.589 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:16.579838+0000 2023-07-29T13:28:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:16.579838+0000 2023-07-29T13:28:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:16.591 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7f8867fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8870140230 0x7f8870140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7f886e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f88700afce0 0x7f887013d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.40:46250/0 (socket says 172.21.15.40:46250) 2023-07-29T13:28:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7f886e59c640 1 -- 172.21.15.40:0/3962878499 learned_addr learned my addr 172.21.15.40:0/3962878499 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:28:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7fa83e9da640 10 monclient: get_monmap_and_config 2023-07-29T13:28:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7fa19399d640 10 monclient: get_monmap_and_config 2023-07-29T13:28:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7fa19399d640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7fa19399d640 10 monclient: monmap: 2023-07-29T13:28:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:16.579541+0000 2023-07-29T13:28:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:16.579541+0000 2023-07-29T13:28:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:16.594 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7f8867fff640 10 monclient(hunting): get_auth_request con 0x7f8870140230 auth_method 0 2023-07-29T13:28:16.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7f8867fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7f8867fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7f8867fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7fe30b05a640 1 Processor -- start 2023-07-29T13:28:16.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7fe30b05a640 1 -- start start 2023-07-29T13:28:16.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7fe30b05a640 10 monclient: init 2023-07-29T13:28:16.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7fa83e9da640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.578+0000 7fa83e9da640 10 monclient: monmap: 2023-07-29T13:28:16.596 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.596 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:16.579258+0000 2023-07-29T13:28:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:16.579258+0000 2023-07-29T13:28:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:16.598 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:16.598 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:16.598 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7f8577fff640 10 monclient: get_monmap_and_config 2023-07-29T13:28:16.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7fa83e9da640 1 Processor -- start 2023-07-29T13:28:16.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7fa83e9da640 1 -- start start 2023-07-29T13:28:16.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7fa83e9da640 10 monclient: init 2023-07-29T13:28:16.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7f8577fff640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7f8577fff640 10 monclient: monmap: 2023-07-29T13:28:16.599 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.600 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.600 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:16.581002+0000 2023-07-29T13:28:16.600 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:16.581002+0000 2023-07-29T13:28:16.600 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.600 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.600 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:16.600 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:16.601 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:16.601 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:16.601 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:16.601 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.579+0000 7f886ed9d640 1 --2- 172.21.15.40:0/3962878499 >> v2:172.21.15.40:3300/0 conn(0x7f8870140b60 0x7f8870147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7f886ed9d640 10 monclient(hunting): get_auth_request con 0x7f8870140b60 auth_method 0 2023-07-29T13:28:16.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7f886ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7f886ed9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7f886ed9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7f8867fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7f8867fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7f8867fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa19399d640 1 Processor -- start 2023-07-29T13:28:16.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7f05a9803640 1 Processor -- start 2023-07-29T13:28:16.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7f05a9803640 1 -- start start 2023-07-29T13:28:16.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7f05a9803640 10 monclient: init 2023-07-29T13:28:16.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2023-07-29T13:28:16.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe3040af910 0x7fe30413bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 10 monclient(hunting): picked mon.noname-c con 0x7fe3040af910 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:16.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe3040afce0 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:16.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe30413ea10 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:16.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3040a9350 con 0x7fe30413ea10 2023-07-29T13:28:16.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30406a070 con 0x7fe3040afce0 2023-07-29T13:28:16.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3040ad660 con 0x7fe3040af910 2023-07-29T13:28:16.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fe30b05a640 10 monclient(hunting): authenticate will time out at 1418.713754s 2023-07-29T13:28:16.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7f0883cbc640 1 Processor -- start 2023-07-29T13:28:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2023-07-29T13:28:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa8380af8e0 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa8380afcb0 0x7fa8380aac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 10 monclient(hunting): picked mon.noname-b con 0x7fa8380afcb0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f0883cbc640 1 -- start start 2023-07-29T13:28:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa8380ae810 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f0883cbc640 10 monclient: init 2023-07-29T13:28:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f886ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f886ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f886ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8380a9f40 con 0x7fa8380af8e0 2023-07-29T13:28:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8380aa220 con 0x7fa8380ae810 2023-07-29T13:28:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f8577fff640 1 Processor -- start 2023-07-29T13:28:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa838073a40 con 0x7fa8380afcb0 2023-07-29T13:28:16.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2023-07-29T13:28:16.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f05a40a86c0 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:16.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa83e9da640 10 monclient(hunting): authenticate will time out at 1418.713797s 2023-07-29T13:28:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa19399d640 1 -- start start 2023-07-29T13:28:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.580+0000 7fa19399d640 10 monclient: init 2023-07-29T13:28:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2023-07-29T13:28:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa18c0afce0 0x7fa18c13d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 10 monclient(hunting): picked mon.noname-a con 0x7fa18c0afce0 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa18c13db50 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa18c140940 0x7fa18c147560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 10 monclient(hunting): picked mon.noname-c con 0x7fa18c140940 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18c0ae960 con 0x7fa18c13db50 2023-07-29T13:28:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c06a070 con 0x7fa18c0afce0 2023-07-29T13:28:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c0ae720 con 0x7fa18c140940 2023-07-29T13:28:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7fa19399d640 10 monclient(hunting): authenticate will time out at 1418.714482s 2023-07-29T13:28:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f05a40aaff0 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05a40ab450 0x7f05a40ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 10 monclient(hunting): picked mon.noname-a con 0x7f05a40ab450 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a40a8a90 con 0x7f05a40aaff0 2023-07-29T13:28:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a40ae6a0 con 0x7f05a40a86c0 2023-07-29T13:28:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a40aa330 con 0x7f05a40ab450 2023-07-29T13:28:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f05a9803640 10 monclient(hunting): authenticate will time out at 1418.714827s 2023-07-29T13:28:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f8577fff640 1 -- start start 2023-07-29T13:28:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.581+0000 7f8577fff640 10 monclient: init 2023-07-29T13:28:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f886dd9b640 1 -- 172.21.15.40:0/3962878499 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128377231 0 0) 0x7f88700ae960 con 0x7f88700afce0 2023-07-29T13:28:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f886dd9b640 10 monclient(hunting): _negotiate 2023-07-29T13:28:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f886dd9b640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f886dd9b640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f886dd9b640 10 monclient(hunting): my global_id is 4285 2023-07-29T13:28:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f886dd9b640 1 -- 172.21.15.40:0/3962878499 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f885c003520 con 0x7f88700afce0 2023-07-29T13:28:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2023-07-29T13:28:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f0883cbc640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f0883cbc640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2023-07-29T13:28:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f85780b6f90 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85780b7360 0x7f8578144380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 10 monclient(hunting): picked mon.noname-c con 0x7f85780b7360 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8578140940 0x7f8578140d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 10 monclient(hunting): picked mon.noname-a con 0x7f8578140940 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85780ae7e0 con 0x7f85780b6f90 2023-07-29T13:28:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f0883cbc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f08740072b0 0x7f0874037250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85780b1060 con 0x7f8578140940 2023-07-29T13:28:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85780aea30 con 0x7f85780b7360 2023-07-29T13:28:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8577fff640 10 monclient(hunting): authenticate will time out at 1418.715695s 2023-07-29T13:28:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f0883cbc640 10 monclient(hunting): picked mon.noname-b con 0x7f08740072b0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f0883cbc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0874034b40 0x7f0874035200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8867fff640 10 monclient(hunting): handle_auth_done global_id 4220 payload 274 2023-07-29T13:28:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8867fff640 1 -- 172.21.15.40:0/3962878499 >> v2:172.21.15.40:3300/0 conn(0x7f8870140b60 msgr2=0x7f8870147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8867fff640 1 --2- 172.21.15.40:0/3962878499 >> v2:172.21.15.40:3300/0 conn(0x7f8870140b60 0x7f8870147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8867fff640 1 -- 172.21.15.40:0/3962878499 >> v1:172.21.15.45:6789/0 conn(0x7f88700afce0 legacy=0x7f887013d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8867fff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8867fff640 1 monclient: found mon.noname-b 2023-07-29T13:28:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8867fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8867fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f8867fff640 1 -- 172.21.15.40:0/3962878499 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88700aa330 con 0x7f8870140230 2023-07-29T13:28:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f0883cbc640 10 monclient(hunting): picked mon.noname-c con 0x7f0874034b40 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7fe30a058640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe3040af910 0x7fe30413bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.582+0000 7f0883cbc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f08740c67b0 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.583+0000 7f0883cbc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0874007680 con 0x7f08740c67b0 2023-07-29T13:28:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.583+0000 7f0883cbc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.583+0000 7f0883cbc640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0874037830 con 0x7f0874034b40 2023-07-29T13:28:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.583+0000 7f0883cbc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.583+0000 7f0883cbc640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0874034f70 con 0x7f08740072b0 2023-07-29T13:28:16.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.583+0000 7f0883cbc640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.583+0000 7f0883cbc640 10 monclient(hunting): authenticate will time out at 1418.716151s 2023-07-29T13:28:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.583+0000 7f0c40dd8640 10 monclient: get_monmap_and_config 2023-07-29T13:28:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.583+0000 7f0c40dd8640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.583+0000 7f0c40dd8640 10 monclient: monmap: 2023-07-29T13:28:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:16.584497+0000 2023-07-29T13:28:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:16.584497+0000 2023-07-29T13:28:16.627 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.627 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.627 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:16.627 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:16.627 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:16.627 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:16.627 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:16.627 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.583+0000 7f0c40dd8640 1 Processor -- start 2023-07-29T13:28:16.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 1 -- start start 2023-07-29T13:28:16.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 10 monclient: init 2023-07-29T13:28:16.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2023-07-29T13:28:16.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0c340072b0 0x7f0c34037250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 10 monclient(hunting): picked mon.noname-c con 0x7f0c340072b0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:16.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c34034b40 0x7f0c34035200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 10 monclient(hunting): picked mon.noname-b con 0x7f0c34034b40 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0c340c67b0 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:16.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c34007680 con 0x7f0c340c67b0 2023-07-29T13:28:16.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c34037830 con 0x7f0c340072b0 2023-07-29T13:28:16.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c34034f70 con 0x7f0c34034b40 2023-07-29T13:28:16.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.584+0000 7f0c40dd8640 10 monclient(hunting): authenticate will time out at 1418.718004s 2023-07-29T13:28:16.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.585+0000 7fa8377fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa8380afcb0 0x7fa8380aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.585+0000 7fe30a058640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe3040af910 0x7fe30413bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:42376/0 (socket says 172.21.15.40:42376) 2023-07-29T13:28:16.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.586+0000 7fe30a058640 1 -- 172.21.15.40:0/3375004302 learned_addr learned my addr 172.21.15.40:0/3375004302 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.586+0000 7fe30a058640 10 monclient(hunting): get_auth_request con 0x7fe3040af910 auth_method 0 2023-07-29T13:28:16.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.586+0000 7fe30a058640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.586+0000 7fe30a058640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.586+0000 7fe30a058640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.586+0000 7f8867fff640 1 --2- 172.21.15.40:0/3962878499 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8870140230 0x7f8870140620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f88600097a0 tx=0x7f886002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17ad9d202f9766d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fa8377fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa8380afcb0 0x7fa8380aac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:48960/0 (socket says 172.21.15.40:48960) 2023-07-29T13:28:16.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fa8377fe640 1 -- 172.21.15.40:0/117637275 learned_addr learned my addr 172.21.15.40:0/117637275 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fa8377fe640 10 monclient(hunting): get_auth_request con 0x7fa8380afcb0 auth_method 0 2023-07-29T13:28:16.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fa8377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fa8377fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fa8377fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fa8377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fa8377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fa8377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fa191739640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa18c0afce0 0x7fa18c13d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fe30a058640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.587+0000 7fe30a058640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.588+0000 7fe30a058640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.588+0000 7f05a9002640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05a40ab450 0x7f05a40ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.589+0000 7f85767fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85780b7360 0x7f8578144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.589+0000 7f0c39d9b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c34034b40 0x7f0c34035200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.589+0000 7f05a9002640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05a40ab450 0x7f05a40ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60490/0 (socket says 172.21.15.40:60490) 2023-07-29T13:28:16.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.589+0000 7f05a9002640 1 -- 172.21.15.40:0/2309327020 learned_addr learned my addr 172.21.15.40:0/2309327020 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.589+0000 7f05a9002640 10 monclient(hunting): get_auth_request con 0x7f05a40ab450 auth_method 0 2023-07-29T13:28:16.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.589+0000 7f05a9002640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.589+0000 7f05a9002640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.589+0000 7f05a9002640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.589+0000 7f85777fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8578140940 0x7f8578140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7f05a9002640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7f05a9002640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7f05a9002640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7f85777fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8578140940 0x7f8578140d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60502/0 (socket says 172.21.15.40:60502) 2023-07-29T13:28:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7f85777fe640 1 -- 172.21.15.40:0/2062781230 learned_addr learned my addr 172.21.15.40:0/2062781230 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7f85777fe640 10 monclient(hunting): get_auth_request con 0x7f8578140940 auth_method 0 2023-07-29T13:28:16.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7f85777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7f85777fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7f85777fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fe30a058640 10 monclient(hunting): handle_auth_done global_id 4288 payload 274 2023-07-29T13:28:16.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fe30a058640 1 -- 172.21.15.40:0/3375004302 >> v1:172.21.15.45:6789/0 conn(0x7fe3040afce0 legacy=0x7fe3040adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fe30a058640 1 -- 172.21.15.40:0/3375004302 >> v1:172.21.15.40:6789/0 conn(0x7fe30413ea10 legacy=0x7fe304149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fe30a058640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fe30a058640 1 monclient: found mon.noname-c 2023-07-29T13:28:16.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fe30a058640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fe30a058640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:28:16.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fe30a058640 1 -- 172.21.15.40:0/3375004302 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3040ae560 con 0x7fe3040af910 2023-07-29T13:28:16.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fa191739640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa18c0afce0 0x7fa18c13d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60480/0 (socket says 172.21.15.40:60480) 2023-07-29T13:28:16.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fa191739640 1 -- 172.21.15.40:0/770249557 learned_addr learned my addr 172.21.15.40:0/770249557 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fa191739640 10 monclient(hunting): get_auth_request con 0x7fa18c0afce0 auth_method 0 2023-07-29T13:28:16.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fa191739640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fa191739640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.590+0000 7fa191739640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.591+0000 7f85767fc640 10 monclient(hunting): get_auth_request con 0x7f85780b7360 auth_method 0 2023-07-29T13:28:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.591+0000 7f85767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.591+0000 7f85767fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.591+0000 7f85767fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7fa8377fe640 10 monclient(hunting): handle_auth_done global_id 4223 payload 274 2023-07-29T13:28:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7fa8377fe640 1 -- 172.21.15.40:0/117637275 >> v1:172.21.15.45:6789/0 conn(0x7fa8380ae810 legacy=0x7fa83814f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7fa8377fe640 1 -- 172.21.15.40:0/117637275 >> v1:172.21.15.40:6789/0 conn(0x7fa8380af8e0 legacy=0x7fa838145960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7fa8377fe640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7fa8377fe640 1 monclient: found mon.noname-b 2023-07-29T13:28:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7fa8377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7fa8377fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7fa8377fe640 1 -- 172.21.15.40:0/117637275 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8380aa500 con 0x7fa8380afcb0 2023-07-29T13:28:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7f0881a58640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f08740072b0 0x7f0874037250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7f05a9002640 10 monclient(hunting): handle_auth_done global_id 4299 payload 274 2023-07-29T13:28:16.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7f05a9002640 1 -- 172.21.15.40:0/2309327020 >> v1:172.21.15.45:6789/0 conn(0x7f05a40a86c0 legacy=0x7f05a40aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7f05a9002640 1 -- 172.21.15.40:0/2309327020 >> v1:172.21.15.40:6789/0 conn(0x7f05a40aaff0 legacy=0x7f05a40ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7f05a9002640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7f05a9002640 1 monclient: found mon.noname-a 2023-07-29T13:28:16.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7f05a9002640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7f05a9002640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.592+0000 7f05a9002640 1 -- 172.21.15.40:0/2309327020 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a40aa610 con 0x7f05a40ab450 2023-07-29T13:28:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0881a58640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f08740072b0 0x7f0874037250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:48970/0 (socket says 172.21.15.40:48970) 2023-07-29T13:28:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0881a58640 1 -- 172.21.15.40:0/1563742300 learned_addr learned my addr 172.21.15.40:0/1563742300 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0881a58640 10 monclient(hunting): get_auth_request con 0x7f08740072b0 auth_method 0 2023-07-29T13:28:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0881a58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0881a58640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0881a58640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f85767fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f85767fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f85767fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0c39d9b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c34034b40 0x7f0c34035200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:48986/0 (socket says 172.21.15.40:48986) 2023-07-29T13:28:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0c39d9b640 1 -- 172.21.15.40:0/2899947560 learned_addr learned my addr 172.21.15.40:0/2899947560 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0c39d9b640 10 monclient(hunting): get_auth_request con 0x7f0c34034b40 auth_method 0 2023-07-29T13:28:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0c39d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0c39d9b640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.593+0000 7f0c39d9b640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.594+0000 7f85777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.594+0000 7f85777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.594+0000 7f85777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.594+0000 7f0881a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.594+0000 7f0881a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.594+0000 7f0881a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7fa8377fe640 1 --2- 172.21.15.40:0/117637275 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa8380afcb0 0x7fa8380aac00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0098a0 tx=0x7fa82c02b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e73cb03ae5d330d server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7f85767fc640 10 monclient(hunting): handle_auth_done global_id 4291 payload 274 2023-07-29T13:28:16.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7f85767fc640 1 -- 172.21.15.40:0/2062781230 >> v2:172.21.15.40:3300/0 conn(0x7f8578140940 msgr2=0x7f8578140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7f85767fc640 1 --2- 172.21.15.40:0/2062781230 >> v2:172.21.15.40:3300/0 conn(0x7f8578140940 0x7f8578140d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7f85767fc640 1 -- 172.21.15.40:0/2062781230 >> v1:172.21.15.45:6789/0 conn(0x7f85780b6f90 legacy=0x7f85780b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7f85767fc640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7f85767fc640 1 monclient: found mon.noname-c 2023-07-29T13:28:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7f85767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7f85767fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:28:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7f85767fc640 1 -- 172.21.15.40:0/2062781230 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85781449f0 con 0x7f85780b7360 2023-07-29T13:28:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7fa191739640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7fa191739640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.595+0000 7fa191739640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.596+0000 7fe30a058640 1 --2- 172.21.15.40:0/3375004302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3040af910 0x7fe30413bfe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009870 tx=0x7fe2f402b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b8bd92ea69042e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.596+0000 7f05a9002640 1 --2- 172.21.15.40:0/2309327020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a40ab450 0x7f05a40ad8b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05a40a8d40 tx=0x7f059c02cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77778ecab6e0b7c server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.596+0000 7fa191f3a640 1 --2- 172.21.15.40:0/770249557 >> v2:172.21.15.45:3300/0 conn(0x7fa18c140940 0x7fa18c147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.597+0000 7f0c39d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.597+0000 7f0c39d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.597+0000 7f0c39d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.597+0000 7f0881257640 1 --2- 172.21.15.40:0/1563742300 >> v2:172.21.15.45:3300/0 conn(0x7f0874034b40 0x7f0874035200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.598+0000 7fa191f3a640 10 monclient(hunting): get_auth_request con 0x7fa18c140940 auth_method 0 2023-07-29T13:28:16.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.598+0000 7fa191f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.598+0000 7fa191f3a640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.598+0000 7fa191f3a640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.598+0000 7f0881257640 10 monclient(hunting): get_auth_request con 0x7f0874034b40 auth_method 0 2023-07-29T13:28:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.598+0000 7f0881257640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.598+0000 7f0881257640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.598+0000 7f0881257640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.599+0000 7fa191f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.599+0000 7fa191f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.599+0000 7fa191f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.599+0000 7f0c3a59c640 1 --2- 172.21.15.40:0/2899947560 >> v2:172.21.15.45:3300/0 conn(0x7f0c340072b0 0x7f0c34037250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.601+0000 7f0881a58640 10 monclient(hunting): handle_auth_done global_id 4226 payload 274 2023-07-29T13:28:16.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.601+0000 7f0881a58640 1 -- 172.21.15.40:0/1563742300 >> v2:172.21.15.45:3300/0 conn(0x7f0874034b40 msgr2=0x7f0874035200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.601+0000 7f0881a58640 1 --2- 172.21.15.40:0/1563742300 >> v2:172.21.15.45:3300/0 conn(0x7f0874034b40 0x7f0874035200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.601+0000 7f0881a58640 1 -- 172.21.15.40:0/1563742300 >> v1:172.21.15.40:6789/0 conn(0x7f08740c67b0 legacy=0x7f0874037d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.601+0000 7f0881a58640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.601+0000 7f0881a58640 1 monclient: found mon.noname-b 2023-07-29T13:28:16.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.601+0000 7f0881a58640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.601+0000 7f0881a58640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.601+0000 7f0881a58640 1 -- 172.21.15.40:0/1563742300 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0874036870 con 0x7f08740072b0 2023-07-29T13:28:16.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.602+0000 7fa191739640 10 monclient(hunting): handle_auth_done global_id 4305 payload 274 2023-07-29T13:28:16.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.602+0000 7fa191739640 1 -- 172.21.15.40:0/770249557 >> v2:172.21.15.45:3300/0 conn(0x7fa18c140940 msgr2=0x7fa18c147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.602+0000 7fa191739640 1 --2- 172.21.15.40:0/770249557 >> v2:172.21.15.45:3300/0 conn(0x7fa18c140940 0x7fa18c147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.602+0000 7fa191739640 1 -- 172.21.15.40:0/770249557 >> v1:172.21.15.40:6789/0 conn(0x7fa18c13db50 legacy=0x7fa18c140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.602+0000 7fa191739640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.602+0000 7fa191739640 1 monclient: found mon.noname-a 2023-07-29T13:28:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.602+0000 7fa191739640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.602+0000 7fa191739640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.602+0000 7fa191739640 1 -- 172.21.15.40:0/770249557 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c0aa330 con 0x7fa18c0afce0 2023-07-29T13:28:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.604+0000 7f0c39d9b640 10 monclient(hunting): handle_auth_done global_id 4229 payload 274 2023-07-29T13:28:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.604+0000 7f0c39d9b640 1 -- 172.21.15.40:0/2899947560 >> v2:172.21.15.45:3300/0 conn(0x7f0c340072b0 msgr2=0x7f0c34037250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.604+0000 7f0c39d9b640 1 --2- 172.21.15.40:0/2899947560 >> v2:172.21.15.45:3300/0 conn(0x7f0c340072b0 0x7f0c34037250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.604+0000 7f0c39d9b640 1 -- 172.21.15.40:0/2899947560 >> v1:172.21.15.45:6789/0 conn(0x7f0c340c67b0 legacy=0x7f0c34037d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.604+0000 7f0c39d9b640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.604+0000 7f0c39d9b640 1 monclient: found mon.noname-b 2023-07-29T13:28:16.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.604+0000 7f0c39d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.604+0000 7f0c39d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.604+0000 7f0c39d9b640 1 -- 172.21.15.40:0/2899947560 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c34036870 con 0x7f0c34034b40 2023-07-29T13:28:16.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.604+0000 7f85767fc640 1 --2- 172.21.15.40:0/2062781230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85780b7360 0x7f8578144380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85780b7ce0 tx=0x7f856c02b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a2adf30809cbf33 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.606+0000 7fa191739640 1 --2- 172.21.15.40:0/770249557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18c0afce0 0x7fa18c13d610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0ac680 tx=0x7fa17c02ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff5f6e81a75ceed server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 1 -- 172.21.15.40:0/117637275 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c031520 con 0x7fa8380afcb0 2023-07-29T13:28:16.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:16.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 10 monclient: dump: 2023-07-29T13:28:16.662 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.662 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.662 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.662 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.662 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.663 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.663 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.663 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.663 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.663 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 1 -- 172.21.15.40:0/117637275 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c031680 con 0x7fa8380afcb0 2023-07-29T13:28:16.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa83e9da640 5 monclient: authenticate success, global_id 4223 2023-07-29T13:28:16.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa83e9da640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:16.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa83e9da640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:16.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa83e9da640 4 set_mon_vals no callback set 2023-07-29T13:28:16.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 1 -- 172.21.15.40:0/117637275 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c031a10 con 0x7fa8380afcb0 2023-07-29T13:28:16.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:16.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa836ffd640 10 monclient: dump: 2023-07-29T13:28:16.665 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.665 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.665 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.665 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.665 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.665 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.666 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.666 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.666 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.666 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa83e9da640 10 monclient: shutdown 2023-07-29T13:28:16.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa83e9da640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa83e9da640 1 -- 172.21.15.40:0/117637275 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa8380afcb0 msgr2=0x7fa8380aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa83e9da640 1 --2- 172.21.15.40:0/117637275 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa8380afcb0 0x7fa8380aac00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0098a0 tx=0x7fa82c02b7c0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa83e9da640 1 -- 172.21.15.40:0/117637275 shutdown_connections 2023-07-29T13:28:16.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa83e9da640 1 --2- 172.21.15.40:0/117637275 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa8380afcb0 0x7fa8380aac00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7f0c39d9b640 1 --2- 172.21.15.40:0/2899947560 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c34034b40 0x7f0c34035200 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009870 tx=0x7f0c3002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f7097deb445728d server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.608+0000 7fa83e9da640 1 -- 172.21.15.40:0/117637275 >> 172.21.15.40:0/117637275 conn(0x7fa83806a2b0 msgr2=0x7fa83806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.609+0000 7fa83e9da640 1 -- 172.21.15.40:0/117637275 shutdown_connections 2023-07-29T13:28:16.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.609+0000 7fa83e9da640 1 -- 172.21.15.40:0/117637275 wait complete. 2023-07-29T13:28:16.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.609+0000 7f0881a58640 1 --2- 172.21.15.40:0/1563742300 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f08740072b0 0x7f0874037250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f086c009870 tx=0x7f086c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e85adb29a87e43d server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.609+0000 7fa83e9da640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.609+0000 7fa83e9da640 10 monclient: monmap: 2023-07-29T13:28:16.670 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.670 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:16.670 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:16.670 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:16.670 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.670 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.670 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:16.671 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.610+0000 7fa83e9da640 1 librados: starting msgr at 2023-07-29T13:28:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.610+0000 7fa83e9da640 1 librados: starting objecter 2023-07-29T13:28:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.610+0000 7fa83e9da640 1 Processor -- start 2023-07-29T13:28:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.610+0000 7fa83e9da640 1 -- start start 2023-07-29T13:28:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.610+0000 7fa83e9da640 1 librados: setting wanted keys 2023-07-29T13:28:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.610+0000 7fa83e9da640 1 librados: calling monclient init 2023-07-29T13:28:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.610+0000 7fa83e9da640 10 monclient: init 2023-07-29T13:28:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T13:28:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8380ae810 0x7fa8380a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 10 monclient(hunting): picked mon.noname-a con 0x7fa8380ae810 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa8380af8e0 0x7fa8381dd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 10 monclient(hunting): picked mon.noname-c con 0x7fa8380af8e0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8380afcb0 0x7fa8381ddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 10 monclient(hunting): picked mon.noname-b con 0x7fa8380afcb0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa838145540 con 0x7fa8380ae810 2023-07-29T13:28:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83806ec40 con 0x7fa8380afcb0 2023-07-29T13:28:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83806dc70 con 0x7fa8380af8e0 2023-07-29T13:28:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.611+0000 7fa83e9da640 10 monclient(hunting): authenticate will time out at 1418.744165s 2023-07-29T13:28:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 1 -- 172.21.15.40:0/3962878499 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886002d040 con 0x7f8870140230 2023-07-29T13:28:16.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:16.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 10 monclient: dump: 2023-07-29T13:28:16.675 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.675 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.675 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.675 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.676 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.676 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.676 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.676 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.676 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.676 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 1 -- 172.21.15.40:0/3962878499 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8860031810 con 0x7f8870140230 2023-07-29T13:28:16.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 1 -- 172.21.15.40:0/3962878499 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8860031ba0 con 0x7f8870140230 2023-07-29T13:28:16.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:16.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f886dd9b640 10 monclient: dump: 2023-07-29T13:28:16.677 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.678 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.678 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.678 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.678 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.678 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.678 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.678 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.678 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.679 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f8874a4f640 5 monclient: authenticate success, global_id 4220 2023-07-29T13:28:16.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f8874a4f640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:16.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.613+0000 7f8874a4f640 4 set_mon_vals no callback set 2023-07-29T13:28:16.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7fa837fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8380ae810 0x7fa8380a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 1 -- 172.21.15.40:0/2309327020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c02d8f0 con 0x7f05a40ab450 2023-07-29T13:28:16.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:16.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 10 monclient: dump: 2023-07-29T13:28:16.681 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.681 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.681 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.681 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.681 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.681 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.681 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.681 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.682 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.682 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 1 -- 172.21.15.40:0/2309327020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c02da50 con 0x7f05a40ab450 2023-07-29T13:28:16.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 1 -- 172.21.15.40:0/2309327020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c02dde0 con 0x7f05a40ab450 2023-07-29T13:28:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a37fe640 10 monclient: dump: 2023-07-29T13:28:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.684 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.684 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.684 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.684 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.684 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a9803640 5 monclient: authenticate success, global_id 4299 2023-07-29T13:28:16.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a9803640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:16.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f05a9803640 4 set_mon_vals no callback set 2023-07-29T13:28:16.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f8874a4f640 10 monclient: shutdown 2023-07-29T13:28:16.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f8874a4f640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f8874a4f640 1 -- 172.21.15.40:0/3962878499 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8870140230 msgr2=0x7f8870140620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f8874a4f640 1 --2- 172.21.15.40:0/3962878499 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8870140230 0x7f8870140620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f88600097a0 tx=0x7f886002b8b0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f8874a4f640 1 -- 172.21.15.40:0/3962878499 shutdown_connections 2023-07-29T13:28:16.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f8874a4f640 1 --2- 172.21.15.40:0/3962878499 >> v2:172.21.15.40:3300/0 conn(0x7f8870140b60 0x7f8870147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f8874a4f640 1 --2- 172.21.15.40:0/3962878499 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8870140230 0x7f8870140620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f8874a4f640 1 -- 172.21.15.40:0/3962878499 >> 172.21.15.40:0/3962878499 conn(0x7f887006a2b0 msgr2=0x7f887006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f8874a4f640 1 -- 172.21.15.40:0/3962878499 shutdown_connections 2023-07-29T13:28:16.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.614+0000 7f8874a4f640 1 -- 172.21.15.40:0/3962878499 wait complete. 2023-07-29T13:28:16.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7fa837fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8380ae810 0x7fa8380a59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60518/0 (socket says 172.21.15.40:60518) 2023-07-29T13:28:16.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7fa837fff640 1 -- 172.21.15.40:0/3317623425 learned_addr learned my addr 172.21.15.40:0/3317623425 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7fa837fff640 10 monclient(hunting): get_auth_request con 0x7fa8380ae810 auth_method 0 2023-07-29T13:28:16.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7fa837fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7fa837fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7fa837fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f05a9803640 10 monclient: shutdown 2023-07-29T13:28:16.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f05a9803640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f05a9803640 1 -- 172.21.15.40:0/2309327020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a40ab450 msgr2=0x7f05a40ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f05a9803640 1 --2- 172.21.15.40:0/2309327020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a40ab450 0x7f05a40ad8b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05a40a8d40 tx=0x7f059c02cf50 comp rx=0 tx=0).stop 2023-07-29T13:28:16.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f05a9803640 1 -- 172.21.15.40:0/2309327020 shutdown_connections 2023-07-29T13:28:16.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f05a9803640 1 --2- 172.21.15.40:0/2309327020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a40ab450 0x7f05a40ad8b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f05a9803640 1 -- 172.21.15.40:0/2309327020 >> 172.21.15.40:0/2309327020 conn(0x7f05a406a2b0 msgr2=0x7f05a406a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f05a9803640 1 -- 172.21.15.40:0/2309327020 shutdown_connections 2023-07-29T13:28:16.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f05a9803640 1 -- 172.21.15.40:0/2309327020 wait complete. 2023-07-29T13:28:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f8874a4f640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f8874a4f640 10 monclient: monmap: 2023-07-29T13:28:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:16.691 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7fa837fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7fa837fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7fa837fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f8874a4f640 1 librados: starting msgr at 2023-07-29T13:28:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f8874a4f640 1 librados: starting objecter 2023-07-29T13:28:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f05a9803640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.615+0000 7f05a9803640 10 monclient: monmap: 2023-07-29T13:28:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:16.694 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.694 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.694 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:16.694 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:16.694 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:16.694 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f8874a4f640 1 Processor -- start 2023-07-29T13:28:16.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f8874a4f640 1 -- start start 2023-07-29T13:28:16.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f8874a4f640 1 librados: setting wanted keys 2023-07-29T13:28:16.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f8874a4f640 1 librados: calling monclient init 2023-07-29T13:28:16.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f8874a4f640 10 monclient: init 2023-07-29T13:28:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f05a9803640 1 librados: starting msgr at 2023-07-29T13:28:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f05a9803640 1 librados: starting objecter 2023-07-29T13:28:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f05a9803640 1 Processor -- start 2023-07-29T13:28:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f05a9803640 1 -- start start 2023-07-29T13:28:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f05a9803640 1 librados: setting wanted keys 2023-07-29T13:28:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f05a9803640 1 librados: calling monclient init 2023-07-29T13:28:16.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f05a9803640 10 monclient: init 2023-07-29T13:28:16.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f8874a4f640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f8874a4f640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T13:28:16.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f8874a4f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f88700afce0 0x7f88700a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.616+0000 7f8874a4f640 10 monclient(hunting): picked mon.noname-c con 0x7f88700afce0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8870140230 0x7f887006f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 10 monclient(hunting): picked mon.noname-a con 0x7f8870140230 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:16.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8870140b60 0x7f887006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 10 monclient(hunting): picked mon.noname-b con 0x7f8870140b60 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:16.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88700a7d30 con 0x7f8870140230 2023-07-29T13:28:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887006f060 con 0x7f8870140b60 2023-07-29T13:28:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88700a5b50 con 0x7f88700afce0 2023-07-29T13:28:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f8874a4f640 10 monclient(hunting): authenticate will time out at 1418.750140s 2023-07-29T13:28:16.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T13:28:16.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40a86c0 0x7f05a41d6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 10 monclient(hunting): picked mon.noname-b con 0x7f05a40a86c0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:16.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f05a40aaff0 0x7f05a41c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 10 monclient(hunting): picked mon.noname-c con 0x7f05a40aaff0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a40ab450 0x7f05a41bed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 10 monclient(hunting): picked mon.noname-a con 0x7f05a40ab450 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:16.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a41a11e0 con 0x7f05a40ab450 2023-07-29T13:28:16.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a41d6ea0 con 0x7f05a40a86c0 2023-07-29T13:28:16.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a41d7350 con 0x7f05a40aaff0 2023-07-29T13:28:16.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7f05a9803640 10 monclient(hunting): authenticate will time out at 1418.750336s 2023-07-29T13:28:16.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.617+0000 7fe309056640 1 -- 172.21.15.40:0/3375004302 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4031750 con 0x7fe3040af910 2023-07-29T13:28:16.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:16.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 10 monclient: dump: 2023-07-29T13:28:16.703 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.703 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.704 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.704 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.704 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.704 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.704 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.704 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.704 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.705 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 1 -- 172.21.15.40:0/3375004302 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f40318b0 con 0x7fe3040af910 2023-07-29T13:28:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 1 -- 172.21.15.40:0/3375004302 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4031c40 con 0x7fe3040af910 2023-07-29T13:28:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe309056640 10 monclient: dump: 2023-07-29T13:28:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.707 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe30b05a640 5 monclient: authenticate success, global_id 4288 2023-07-29T13:28:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe30b05a640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe30b05a640 4 set_mon_vals no callback set 2023-07-29T13:28:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 1 -- 172.21.15.40:0/2062781230 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c031710 con 0x7f85780b7360 2023-07-29T13:28:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 10 monclient: dump: 2023-07-29T13:28:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.710 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.710 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8577fff640 5 monclient: authenticate success, global_id 4291 2023-07-29T13:28:16.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 1 -- 172.21.15.40:0/2062781230 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c02f050 con 0x7f85780b7360 2023-07-29T13:28:16.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8577fff640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 1 -- 172.21.15.40:0/2062781230 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c031a90 con 0x7f85780b7360 2023-07-29T13:28:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8575ffb640 10 monclient: dump: 2023-07-29T13:28:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.712 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8577fff640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:16.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7f8577fff640 4 set_mon_vals no callback set 2023-07-29T13:28:16.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe30b05a640 10 monclient: shutdown 2023-07-29T13:28:16.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe30b05a640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe30b05a640 1 -- 172.21.15.40:0/3375004302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3040af910 msgr2=0x7fe30413bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe30b05a640 1 --2- 172.21.15.40:0/3375004302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3040af910 0x7fe30413bfe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009870 tx=0x7fe2f402b7f0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe30b05a640 1 -- 172.21.15.40:0/3375004302 shutdown_connections 2023-07-29T13:28:16.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe30b05a640 1 --2- 172.21.15.40:0/3375004302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3040af910 0x7fe30413bfe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.618+0000 7fe30b05a640 1 -- 172.21.15.40:0/3375004302 >> 172.21.15.40:0/3375004302 conn(0x7fe30406a2b0 msgr2=0x7fe30406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f8577fff640 10 monclient: shutdown 2023-07-29T13:28:16.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f8577fff640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f8577fff640 1 -- 172.21.15.40:0/2062781230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85780b7360 msgr2=0x7f8578144380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f8577fff640 1 --2- 172.21.15.40:0/2062781230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85780b7360 0x7f8578144380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85780b7ce0 tx=0x7f856c02b7b0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fe30b05a640 1 -- 172.21.15.40:0/3375004302 shutdown_connections 2023-07-29T13:28:16.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fe30b05a640 1 -- 172.21.15.40:0/3375004302 wait complete. 2023-07-29T13:28:16.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f8577fff640 1 -- 172.21.15.40:0/2062781230 shutdown_connections 2023-07-29T13:28:16.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f8577fff640 1 --2- 172.21.15.40:0/2062781230 >> v2:172.21.15.40:3300/0 conn(0x7f8578140940 0x7f8578140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f8577fff640 1 --2- 172.21.15.40:0/2062781230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85780b7360 0x7f8578144380 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f8577fff640 1 -- 172.21.15.40:0/2062781230 >> 172.21.15.40:0/2062781230 conn(0x7f857806a2b0 msgr2=0x7f857806a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 1 -- 172.21.15.40:0/1563742300 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c031640 con 0x7f08740072b0 2023-07-29T13:28:16.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:16.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 10 monclient: dump: 2023-07-29T13:28:16.717 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.717 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.718 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.718 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.718 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.718 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.718 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.718 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.718 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.718 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 1 -- 172.21.15.40:0/1563742300 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c0317a0 con 0x7f08740072b0 2023-07-29T13:28:16.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0883cbc640 5 monclient: authenticate success, global_id 4226 2023-07-29T13:28:16.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 1 -- 172.21.15.40:0/1563742300 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c031b30 con 0x7f08740072b0 2023-07-29T13:28:16.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0883cbc640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:16.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0883cbc640 4 set_mon_vals no callback set 2023-07-29T13:28:16.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:16.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0880a56640 10 monclient: dump: 2023-07-29T13:28:16.720 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.720 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.720 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.721 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.721 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.721 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.721 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.721 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.721 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.721 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fa837fff640 10 monclient(hunting): handle_auth_done global_id 4308 payload 824 2023-07-29T13:28:16.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fa837fff640 1 -- 172.21.15.40:0/3317623425 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa8380af8e0 msgr2=0x7fa8381dd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fa837fff640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa8380af8e0 0x7fa8381dd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fa837fff640 1 -- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8380afcb0 msgr2=0x7fa8381ddbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fa837fff640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8380afcb0 0x7fa8381ddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f8577fff640 1 -- 172.21.15.40:0/2062781230 shutdown_connections 2023-07-29T13:28:16.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fa837fff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fa837fff640 1 monclient: found mon.noname-a 2023-07-29T13:28:16.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fa837fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fa837fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fa837fff640 1 -- 172.21.15.40:0/3317623425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0094d0 con 0x7fa8380ae810 2023-07-29T13:28:16.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0883cbc640 10 monclient: shutdown 2023-07-29T13:28:16.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0883cbc640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0883cbc640 1 -- 172.21.15.40:0/1563742300 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f08740072b0 msgr2=0x7f0874037250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f0883cbc640 1 --2- 172.21.15.40:0/1563742300 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f08740072b0 0x7f0874037250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f086c009870 tx=0x7f086c02b8f0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fe30b05a640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7fe30b05a640 10 monclient: monmap: 2023-07-29T13:28:16.725 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.725 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:16.725 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:16.725 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:16.726 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.726 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.726 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:16.726 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:16.726 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:16.726 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7f0883cbc640 1 -- 172.21.15.40:0/1563742300 shutdown_connections 2023-07-29T13:28:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7f0883cbc640 1 --2- 172.21.15.40:0/1563742300 >> v2:172.21.15.45:3300/0 conn(0x7f0874034b40 0x7f0874035200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7f0883cbc640 1 --2- 172.21.15.40:0/1563742300 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f08740072b0 0x7f0874037250 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7f0883cbc640 1 -- 172.21.15.40:0/1563742300 >> 172.21.15.40:0/1563742300 conn(0x7f0874039e90 msgr2=0x7f087403a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.619+0000 7f8577fff640 1 -- 172.21.15.40:0/2062781230 wait complete. 2023-07-29T13:28:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7fe30b05a640 1 librados: starting msgr at 2023-07-29T13:28:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7fe30b05a640 1 librados: starting objecter 2023-07-29T13:28:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7fe30b05a640 1 Processor -- start 2023-07-29T13:28:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7f0883cbc640 1 -- 172.21.15.40:0/1563742300 shutdown_connections 2023-07-29T13:28:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7f0883cbc640 1 -- 172.21.15.40:0/1563742300 wait complete. 2023-07-29T13:28:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7fe30b05a640 1 -- start start 2023-07-29T13:28:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7fe30b05a640 1 librados: setting wanted keys 2023-07-29T13:28:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7fe30b05a640 1 librados: calling monclient init 2023-07-29T13:28:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7fe30b05a640 10 monclient: init 2023-07-29T13:28:16.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0883cbc640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0883cbc640 10 monclient: monmap: 2023-07-29T13:28:16.730 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.730 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:16.730 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:16.731 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:16.731 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.731 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.731 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:16.731 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:16.731 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:16.731 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0883cbc640 1 librados: starting msgr at 2023-07-29T13:28:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0883cbc640 1 librados: starting objecter 2023-07-29T13:28:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7f8577fff640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.620+0000 7f8577fff640 10 monclient: monmap: 2023-07-29T13:28:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:16.733 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f8577fff640 1 librados: starting msgr at 2023-07-29T13:28:16.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f8577fff640 1 librados: starting objecter 2023-07-29T13:28:16.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f8577fff640 1 Processor -- start 2023-07-29T13:28:16.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T13:28:16.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0883cbc640 1 Processor -- start 2023-07-29T13:28:16.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f8577fff640 1 -- start start 2023-07-29T13:28:16.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f8577fff640 1 librados: setting wanted keys 2023-07-29T13:28:16.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f8577fff640 1 librados: calling monclient init 2023-07-29T13:28:16.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f8577fff640 10 monclient: init 2023-07-29T13:28:16.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3040af910 0x7fe30413a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 10 monclient(hunting): picked mon.noname-b con 0x7fe3040af910 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:16.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe3040afce0 0x7fe3040a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 10 monclient(hunting): picked mon.noname-c con 0x7fe3040afce0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe30413ea10 0x7fe3040b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 10 monclient(hunting): picked mon.noname-a con 0x7fe30413ea10 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:16.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30413a190 con 0x7fe30413ea10 2023-07-29T13:28:16.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30413bce0 con 0x7fe3040af910 2023-07-29T13:28:16.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30413bfc0 con 0x7fe3040afce0 2023-07-29T13:28:16.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7fe30b05a640 10 monclient(hunting): authenticate will time out at 1418.754576s 2023-07-29T13:28:16.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 1 -- 172.21.15.40:0/2899947560 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30031640 con 0x7f0c34034b40 2023-07-29T13:28:16.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:16.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 10 monclient: dump: 2023-07-29T13:28:16.738 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.738 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.739 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.739 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.739 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.739 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.739 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.739 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.739 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.739 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 1 -- 172.21.15.40:0/2899947560 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c300317a0 con 0x7f0c34034b40 2023-07-29T13:28:16.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 1 -- 172.21.15.40:0/2899947560 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30031b30 con 0x7f0c34034b40 2023-07-29T13:28:16.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:16.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c3959a640 10 monclient: dump: 2023-07-29T13:28:16.741 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.741 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.741 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.741 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.741 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.741 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.741 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.742 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.742 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.742 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0883cbc640 1 -- start start 2023-07-29T13:28:16.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0883cbc640 1 librados: setting wanted keys 2023-07-29T13:28:16.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0883cbc640 1 librados: calling monclient init 2023-07-29T13:28:16.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0883cbc640 10 monclient: init 2023-07-29T13:28:16.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c40dd8640 5 monclient: authenticate success, global_id 4229 2023-07-29T13:28:16.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c40dd8640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:16.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.621+0000 7f0c40dd8640 4 set_mon_vals no callback set 2023-07-29T13:28:16.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T13:28:16.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85780b6f90 0x7f857806cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 10 monclient(hunting): picked mon.noname-a con 0x7f85780b6f90 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:16.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85780b7360 0x7f857813d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 10 monclient(hunting): picked mon.noname-b con 0x7f85780b7360 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:16.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8578140940 0x7f857813dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 10 monclient(hunting): picked mon.noname-c con 0x7f8578140940 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857806d7f0 con 0x7f85780b6f90 2023-07-29T13:28:16.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857813e210 con 0x7f85780b7360 2023-07-29T13:28:16.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857806c870 con 0x7f8578140940 2023-07-29T13:28:16.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f8577fff640 10 monclient(hunting): authenticate will time out at 1418.755477s 2023-07-29T13:28:16.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T13:28:16.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08740072b0 0x7f087402feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 10 monclient(hunting): picked mon.noname-a con 0x7f08740072b0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:16.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0874034b40 0x7f087415fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 10 monclient(hunting): picked mon.noname-c con 0x7f0874034b40 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08740c67b0 0x7f08741601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 10 monclient(hunting): picked mon.noname-b con 0x7f08740c67b0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:16.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087415c1f0 con 0x7f08740072b0 2023-07-29T13:28:16.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740c9070 con 0x7f08740c67b0 2023-07-29T13:28:16.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087402f000 con 0x7f0874034b40 2023-07-29T13:28:16.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0883cbc640 10 monclient(hunting): authenticate will time out at 1418.755533s 2023-07-29T13:28:16.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0c40dd8640 10 monclient: shutdown 2023-07-29T13:28:16.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0c40dd8640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0c40dd8640 1 -- 172.21.15.40:0/2899947560 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c34034b40 msgr2=0x7f0c34035200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0c40dd8640 1 --2- 172.21.15.40:0/2899947560 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c34034b40 0x7f0c34035200 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009870 tx=0x7f0c3002b8f0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0c40dd8640 1 -- 172.21.15.40:0/2899947560 shutdown_connections 2023-07-29T13:28:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0c40dd8640 1 --2- 172.21.15.40:0/2899947560 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c34034b40 0x7f0c34035200 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0c40dd8640 1 --2- 172.21.15.40:0/2899947560 >> v2:172.21.15.45:3300/0 conn(0x7f0c340072b0 0x7f0c34037250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0c40dd8640 1 -- 172.21.15.40:0/2899947560 >> 172.21.15.40:0/2899947560 conn(0x7f0c34039e90 msgr2=0x7f0c3403a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0c40dd8640 1 -- 172.21.15.40:0/2899947560 shutdown_connections 2023-07-29T13:28:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.622+0000 7f0c40dd8640 1 -- 172.21.15.40:0/2899947560 wait complete. 2023-07-29T13:28:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.623+0000 7f0c40dd8640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.623+0000 7f0c40dd8640 10 monclient: monmap: 2023-07-29T13:28:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:16.752 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 librados: starting msgr at 2023-07-29T13:28:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 librados: starting objecter 2023-07-29T13:28:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 Processor -- start 2023-07-29T13:28:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 -- start start 2023-07-29T13:28:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 librados: setting wanted keys 2023-07-29T13:28:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 librados: calling monclient init 2023-07-29T13:28:16.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 10 monclient: init 2023-07-29T13:28:16.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f886e59c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f88700afce0 0x7f88700a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T13:28:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c340072b0 0x7f0c3402e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 10 monclient(hunting): picked mon.noname-c con 0x7f0c340072b0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c34034b40 0x7f0c340321e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 10 monclient(hunting): picked mon.noname-a con 0x7f0c34034b40 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c340c67b0 0x7f0c3402f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 10 monclient(hunting): picked mon.noname-b con 0x7f0c340c67b0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c34032c80 con 0x7f0c34034b40 2023-07-29T13:28:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3402fe60 con 0x7f0c340c67b0 2023-07-29T13:28:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3402f940 con 0x7f0c340072b0 2023-07-29T13:28:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.624+0000 7f0c40dd8640 10 monclient(hunting): authenticate will time out at 1418.757991s 2023-07-29T13:28:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.625+0000 7f886e59c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f88700afce0 0x7f88700a5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49002/0 (socket says 172.21.15.40:49002) 2023-07-29T13:28:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.625+0000 7f886e59c640 1 -- 172.21.15.40:0/1599888459 learned_addr learned my addr 172.21.15.40:0/1599888459 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.625+0000 7f886e59c640 10 monclient(hunting): get_auth_request con 0x7f88700afce0 auth_method 0 2023-07-29T13:28:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.625+0000 7f886e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.625+0000 7f886e59c640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.625+0000 7f886e59c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.625+0000 7f05a8801640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40a86c0 0x7f05a41d6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.626+0000 7fa837fff640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8380ae810 0x7fa8380a59d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa82800c410 tx=0x7fa82800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.626+0000 7fa834ff9640 10 client.?.objecter ms_handle_connect 0x7fa8380ae810 2023-07-29T13:28:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.626+0000 7fa834ff9640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.626+0000 7f05a8801640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40a86c0 0x7f05a41d6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:42450/0 (socket says 172.21.15.40:42450) 2023-07-29T13:28:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.626+0000 7f05a8801640 1 -- 172.21.15.40:0/4007714830 learned_addr learned my addr 172.21.15.40:0/4007714830 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.626+0000 7f05a8801640 10 monclient(hunting): get_auth_request con 0x7f05a40a86c0 auth_method 0 2023-07-29T13:28:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.626+0000 7f05a8801640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.626+0000 7f05a8801640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.626+0000 7f05a8801640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 1 -- 172.21.15.40:0/770249557 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c008df0 con 0x7fa18c0afce0 2023-07-29T13:28:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:16.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 10 monclient: dump: 2023-07-29T13:28:16.760 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.760 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.760 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.760 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.760 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.760 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.761 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 1 -- 172.21.15.40:0/770249557 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c004030 con 0x7fa18c0afce0 2023-07-29T13:28:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 1 -- 172.21.15.40:0/770249557 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c004420 con 0x7fa18c0afce0 2023-07-29T13:28:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa19399d640 5 monclient: authenticate success, global_id 4305 2023-07-29T13:28:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa19399d640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa19399d640 4 set_mon_vals no callback set 2023-07-29T13:28:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa183fff640 10 monclient: dump: 2023-07-29T13:28:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.764 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fe309857640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe3040afce0 0x7fe3040a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7f05a3fff640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f05a40aaff0 0x7f05a41c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa19399d640 10 monclient: shutdown 2023-07-29T13:28:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa19399d640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa19399d640 1 -- 172.21.15.40:0/770249557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18c0afce0 msgr2=0x7fa18c13d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa19399d640 1 --2- 172.21.15.40:0/770249557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18c0afce0 0x7fa18c13d610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0ac680 tx=0x7fa17c02ca00 comp rx=0 tx=0).stop 2023-07-29T13:28:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa19399d640 1 -- 172.21.15.40:0/770249557 shutdown_connections 2023-07-29T13:28:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa19399d640 1 --2- 172.21.15.40:0/770249557 >> v2:172.21.15.45:3300/0 conn(0x7fa18c140940 0x7fa18c147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa19399d640 1 --2- 172.21.15.40:0/770249557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18c0afce0 0x7fa18c13d610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.627+0000 7fa19399d640 1 -- 172.21.15.40:0/770249557 >> 172.21.15.40:0/770249557 conn(0x7fa18c06a2b0 msgr2=0x7fa18c06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7fa19399d640 1 -- 172.21.15.40:0/770249557 shutdown_connections 2023-07-29T13:28:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7fe309857640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe3040afce0 0x7fe3040a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49032/0 (socket says 172.21.15.40:49032) 2023-07-29T13:28:16.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7fe309857640 1 -- 172.21.15.40:0/1622460967 learned_addr learned my addr 172.21.15.40:0/1622460967 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7fe309857640 10 monclient(hunting): get_auth_request con 0x7fe3040afce0 auth_method 0 2023-07-29T13:28:16.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7fe309857640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7fe309857640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7fe309857640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7f05a3fff640 10 monclient(hunting): get_auth_request con 0x7f05a40aaff0 auth_method 0 2023-07-29T13:28:16.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7f05a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7f05a3fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7f05a3fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7f886e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7f886e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7f886e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7fa19399d640 1 -- 172.21.15.40:0/770249557 wait complete. 2023-07-29T13:28:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7f05a8801640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7f05a8801640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7f05a8801640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.628+0000 7f85767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85780b7360 0x7f857813d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fa19399d640 10 monclient: build_initial_monmap 2023-07-29T13:28:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fa19399d640 10 monclient: monmap: 2023-07-29T13:28:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:16.772 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:16.772 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:16.772 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fe309857640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fe309857640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fe309857640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fa19399d640 1 librados: starting msgr at 2023-07-29T13:28:16.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fa19399d640 1 librados: starting objecter 2023-07-29T13:28:16.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7f886ed9d640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8870140b60 0x7f887006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fa19399d640 1 Processor -- start 2023-07-29T13:28:16.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fa19399d640 1 -- start start 2023-07-29T13:28:16.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fa19399d640 1 librados: setting wanted keys 2023-07-29T13:28:16.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fa19399d640 1 librados: calling monclient init 2023-07-29T13:28:16.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.629+0000 7fa19399d640 10 monclient: init 2023-07-29T13:28:16.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:16.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 10 monclient: _add_conns ranks=[1,0,2] 2023-07-29T13:28:16.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18c0afce0 0x7fa18c0a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 10 monclient(hunting): picked mon.noname-b con 0x7fa18c0afce0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:16.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18c13db50 0x7fa18c1d9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 10 monclient(hunting): picked mon.noname-a con 0x7fa18c13db50 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:16.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa18c140940 0x7fa18c070b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 10 monclient(hunting): picked mon.noname-c con 0x7fa18c140940 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:16.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0a3010 con 0x7fa18c13db50 2023-07-29T13:28:16.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c1d9cf0 con 0x7fa18c0afce0 2023-07-29T13:28:16.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:16.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c1d9760 con 0x7fa18c140940 2023-07-29T13:28:16.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:16.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.630+0000 7fa19399d640 10 monclient(hunting): authenticate will time out at 1418.763697s 2023-07-29T13:28:16.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f05a8801640 10 monclient(hunting): handle_auth_done global_id 4300 payload 824 2023-07-29T13:28:16.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f05a8801640 1 -- 172.21.15.40:0/4007714830 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f05a40aaff0 msgr2=0x7f05a41c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f05a8801640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f05a40aaff0 0x7f05a41c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f05a8801640 1 -- 172.21.15.40:0/4007714830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a40ab450 msgr2=0x7f05a41bed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f05a8801640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a40ab450 0x7f05a41bed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f05a8801640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f05a8801640 1 monclient: found mon.noname-b 2023-07-29T13:28:16.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f05a8801640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f05a8801640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:16.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f05a8801640 1 -- 172.21.15.40:0/4007714830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c00ac40 con 0x7f05a40a86c0 2023-07-29T13:28:16.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f886e59c640 10 monclient(hunting): handle_auth_done global_id 4232 payload 824 2023-07-29T13:28:16.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f886e59c640 1 -- 172.21.15.40:0/1599888459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8870140b60 msgr2=0x7f887006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f886e59c640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8870140b60 0x7f887006d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f886e59c640 1 -- 172.21.15.40:0/1599888459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8870140230 msgr2=0x7f887006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f886e59c640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8870140230 0x7f887006f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f886e59c640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f886e59c640 1 monclient: found mon.noname-c 2023-07-29T13:28:16.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f886e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f886e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f886e59c640 1 -- 172.21.15.40:0/1599888459 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8860009400 con 0x7f88700afce0 2023-07-29T13:28:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f85767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85780b7360 0x7f857813d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:42476/0 (socket says 172.21.15.40:42476) 2023-07-29T13:28:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f85767fc640 1 -- 172.21.15.40:0/3533042190 learned_addr learned my addr 172.21.15.40:0/3533042190 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f85767fc640 10 monclient(hunting): get_auth_request con 0x7f85780b7360 auth_method 0 2023-07-29T13:28:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f85767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f85767fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.632+0000 7f85767fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.634+0000 7f85767fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.634+0000 7f85767fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.634+0000 7f85767fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7fe30a859640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe30413ea10 0x7fe3040b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7fe309857640 10 monclient(hunting): handle_auth_done global_id 4235 payload 824 2023-07-29T13:28:16.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7f85767fc640 10 monclient(hunting): handle_auth_done global_id 4303 payload 824 2023-07-29T13:28:16.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7fe309857640 1 -- 172.21.15.40:0/1622460967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3040af910 msgr2=0x7fe30413a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7fe309857640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3040af910 0x7fe30413a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7fe309857640 1 -- 172.21.15.40:0/1622460967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe30413ea10 msgr2=0x7fe3040b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7fe309857640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe30413ea10 0x7fe3040b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7fe309857640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7fe309857640 1 monclient: found mon.noname-c 2023-07-29T13:28:16.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7fe309857640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7fe309857640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7fe309857640 1 -- 172.21.15.40:0/1622460967 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f40094d0 con 0x7fe3040afce0 2023-07-29T13:28:16.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7f85767fc640 1 -- 172.21.15.40:0/3533042190 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8578140940 msgr2=0x7f857813dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7f85767fc640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8578140940 0x7f857813dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7f85767fc640 1 -- 172.21.15.40:0/3533042190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85780b6f90 msgr2=0x7f857806cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7f85767fc640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85780b6f90 0x7f857806cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7f85767fc640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7f85767fc640 1 monclient: found mon.noname-b 2023-07-29T13:28:16.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7f85767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7f85767fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:16.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.637+0000 7f85767fc640 1 -- 172.21.15.40:0/3533042190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c009400 con 0x7f85780b7360 2023-07-29T13:28:16.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.638+0000 7f05a8801640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40a86c0 0x7f05a41d6960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f059400c410 tx=0x7f059400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.638+0000 7f05a17fa640 10 client.?.objecter ms_handle_connect 0x7f05a40a86c0 2023-07-29T13:28:16.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.638+0000 7f05a17fa640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:16.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.638+0000 7f0c3a59c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c340072b0 0x7f0c3402e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.639+0000 7f0c3a59c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c340072b0 0x7f0c3402e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49054/0 (socket says 172.21.15.40:49054) 2023-07-29T13:28:16.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.639+0000 7f0c3a59c640 1 -- 172.21.15.40:0/762864763 learned_addr learned my addr 172.21.15.40:0/762864763 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.639+0000 7f0c3a59c640 10 monclient(hunting): get_auth_request con 0x7f0c340072b0 auth_method 0 2023-07-29T13:28:16.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.639+0000 7f0c3a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.639+0000 7f0c3a59c640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.639+0000 7f0c3a59c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.641+0000 7f0882259640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08740c67b0 0x7f08741601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.641+0000 7f0c3a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.641+0000 7f0c3a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.641+0000 7f0c3a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.641+0000 7f886e59c640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f88700afce0 0x7f88700a5090 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f885800c7f0 tx=0x7f885800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.641+0000 7f8866ffd640 10 client.?.objecter ms_handle_connect 0x7f88700afce0 2023-07-29T13:28:16.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.641+0000 7f8866ffd640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:16.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.641+0000 7f85767fc640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85780b7360 0x7f857813d790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f856c009890 tx=0x7f856c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.641+0000 7f855ffff640 10 client.?.objecter ms_handle_connect 0x7f85780b7360 2023-07-29T13:28:16.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.641+0000 7f855ffff640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:16.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 1 -- 172.21.15.40:0/3317623425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280035f0 con 0x7fa8380ae810 2023-07-29T13:28:16.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:16.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 10 monclient: dump: 2023-07-29T13:28:16.791 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.791 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.791 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.791 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.791 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.791 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.792 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.792 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.792 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.792 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 1 -- 172.21.15.40:0/3317623425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa828003750 con 0x7fa8380ae810 2023-07-29T13:28:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 1 -- 172.21.15.40:0/3317623425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828003b40 con 0x7fa8380ae810 2023-07-29T13:28:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa834ff9640 10 monclient: dump: 2023-07-29T13:28:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.794 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa8367fc640 4 set_mon_vals no callback set 2023-07-29T13:28:16.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa190f38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18c13db50 0x7fa18c1d9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa83e9da640 5 monclient: authenticate success, global_id 4308 2023-07-29T13:28:16.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa83e9da640 10 monclient: _renew_subs 2023-07-29T13:28:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa83e9da640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa83e9da640 1 -- 172.21.15.40:0/3317623425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83806fb70 con 0x7fa8380ae810 2023-07-29T13:28:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa83e9da640 10 client.4308.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa83e9da640 10 monclient: _renew_subs 2023-07-29T13:28:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa83e9da640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa83e9da640 1 -- 172.21.15.40:0/3317623425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8381de170 con 0x7fa8380ae810 2023-07-29T13:28:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7fa83e9da640 1 librados: init done 2023-07-29T13:28:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.643+0000 7f0c39d9b640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c34034b40 0x7f0c340321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7f0881a58640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08740072b0 0x7f087402feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7f0881a58640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08740072b0 0x7f087402feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60564/0 (socket says 172.21.15.40:60564) 2023-07-29T13:28:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7f0881a58640 1 -- 172.21.15.40:0/2431520686 learned_addr learned my addr 172.21.15.40:0/2431520686 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7f0881a58640 10 monclient(hunting): get_auth_request con 0x7f08740072b0 auth_method 0 2023-07-29T13:28:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7f0881a58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7f0881a58640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7f0881a58640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7fa83e9da640 10 client.4308.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7fa83e9da640 15 client.4308.objecter _session_command_op_assign -1 1 2023-07-29T13:28:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7fa83e9da640 20 client.4308.objecter _get_session osd=-1 returning homeless 2023-07-29T13:28:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7fa83e9da640 10 client.4308.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7fa83e9da640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7fa83e9da640 1 -- 172.21.15.40:0/3317623425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8380ab140 con 0x7fa8380ae810 2023-07-29T13:28:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7f0c3ad9d640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c340c67b0 0x7f0c3402f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7f0c3a59c640 10 monclient(hunting): handle_auth_done global_id 4241 payload 824 2023-07-29T13:28:16.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7f0881a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.644+0000 7f0881a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7f0881a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7f0c3a59c640 1 -- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c340c67b0 msgr2=0x7f0c3402f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7f0c3a59c640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c340c67b0 0x7f0c3402f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7f0c3a59c640 1 -- 172.21.15.40:0/762864763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c34034b40 msgr2=0x7f0c340321e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7f0c3a59c640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c34034b40 0x7f0c340321e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7f0c3a59c640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7f0c3a59c640 1 monclient: found mon.noname-c 2023-07-29T13:28:16.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7f0c3a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7f0c3a59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7f0c3a59c640 1 -- 172.21.15.40:0/762864763 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c300094d0 con 0x7f0c340072b0 2023-07-29T13:28:16.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7fa190f38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18c13db50 0x7fa18c1d9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60586/0 (socket says 172.21.15.40:60586) 2023-07-29T13:28:16.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7fa190f38640 1 -- 172.21.15.40:0/4148855278 learned_addr learned my addr 172.21.15.40:0/4148855278 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7fa190f38640 10 monclient(hunting): get_auth_request con 0x7fa18c13db50 auth_method 0 2023-07-29T13:28:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7fa190f38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7fa190f38640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.645+0000 7fa190f38640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.647+0000 7f0882259640 10 monclient(hunting): get_auth_request con 0x7f08740c67b0 auth_method 0 2023-07-29T13:28:16.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.647+0000 7f0882259640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:16.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.647+0000 7f0882259640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:16.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.647+0000 7f0882259640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:16.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.648+0000 7f0881a58640 10 monclient(hunting): handle_auth_done global_id 4311 payload 824 2023-07-29T13:28:16.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.648+0000 7f0881a58640 1 -- 172.21.15.40:0/2431520686 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0874034b40 msgr2=0x7f087415fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.648+0000 7f0881a58640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0874034b40 0x7f087415fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.648+0000 7f0881a58640 1 -- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08740c67b0 msgr2=0x7f08741601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.648+0000 7f0881a58640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08740c67b0 0x7f08741601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.648+0000 7f0881a58640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.648+0000 7f0881a58640 1 monclient: found mon.noname-a 2023-07-29T13:28:16.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.648+0000 7f0881a58640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.648+0000 7f0881a58640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.648+0000 7f0881a58640 1 -- 172.21.15.40:0/2431520686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c0094d0 con 0x7f08740072b0 2023-07-29T13:28:16.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.649+0000 7fa190f38640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:16.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.649+0000 7fa190f38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:16.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.649+0000 7fa190f38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:16.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.649+0000 7fe309857640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe3040afce0 0x7fe3040a4e70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00c7f0 tx=0x7fe2fc00ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.649+0000 7fe2faffd640 10 client.?.objecter ms_handle_connect 0x7fe3040afce0 2023-07-29T13:28:16.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.649+0000 7fe2faffd640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:16.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 1 -- 172.21.15.40:0/4007714830 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05940035f0 con 0x7f05a40a86c0 2023-07-29T13:28:16.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:16.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 10 monclient: dump: 2023-07-29T13:28:16.808 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.808 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.808 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.808 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.808 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.809 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.809 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.809 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.809 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.809 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 1 -- 172.21.15.40:0/4007714830 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0594003750 con 0x7f05a40a86c0 2023-07-29T13:28:16.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 1 -- 172.21.15.40:0/4007714830 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594003b00 con 0x7f05a40a86c0 2023-07-29T13:28:16.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:16.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a17fa640 10 monclient: dump: 2023-07-29T13:28:16.810 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.810 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.811 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.811 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.811 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.811 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.811 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.811 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.811 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.811 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a9803640 5 monclient: authenticate success, global_id 4300 2023-07-29T13:28:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a9803640 10 monclient: _renew_subs 2023-07-29T13:28:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a9803640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a9803640 1 -- 172.21.15.40:0/4007714830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a41d75d0 con 0x7f05a40a86c0 2023-07-29T13:28:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a2ffd640 4 set_mon_vals no callback set 2023-07-29T13:28:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a9803640 10 client.4300.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a9803640 10 monclient: _renew_subs 2023-07-29T13:28:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a9803640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a9803640 1 -- 172.21.15.40:0/4007714830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a41bfaf0 con 0x7f05a40a86c0 2023-07-29T13:28:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.651+0000 7f05a9803640 1 librados: init done 2023-07-29T13:28:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.652+0000 7f0c3a59c640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c340072b0 0x7f0c3402e880 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c820 tx=0x7f0c2c00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.652+0000 7f0c2affd640 10 client.?.objecter ms_handle_connect 0x7f0c340072b0 2023-07-29T13:28:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.652+0000 7f0c2affd640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.652+0000 7f05a9803640 10 client.4300.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.652+0000 7f05a9803640 15 client.4300.objecter _session_command_op_assign -1 1 2023-07-29T13:28:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.652+0000 7f05a9803640 20 client.4300.objecter _get_session osd=-1 returning homeless 2023-07-29T13:28:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.652+0000 7f05a9803640 10 client.4300.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.652+0000 7f05a9803640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.652+0000 7f05a9803640 1 -- 172.21.15.40:0/4007714830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f05a40aabd0 con 0x7f05a40a86c0 2023-07-29T13:28:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.653+0000 7fa190f38640 10 monclient(hunting): handle_auth_done global_id 4314 payload 824 2023-07-29T13:28:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.653+0000 7fa190f38640 1 -- 172.21.15.40:0/4148855278 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa18c140940 msgr2=0x7fa18c070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.653+0000 7fa190f38640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa18c140940 0x7fa18c070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.653+0000 7fa190f38640 1 -- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18c0afce0 msgr2=0x7fa18c0a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.653+0000 7fa190f38640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18c0afce0 0x7fa18c0a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.653+0000 7fa190f38640 10 monclient: _finish_hunting 0 2023-07-29T13:28:16.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.653+0000 7fa190f38640 1 monclient: found mon.noname-a 2023-07-29T13:28:16.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.653+0000 7fa190f38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:16.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.653+0000 7fa190f38640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.653+0000 7fa190f38640 1 -- 172.21.15.40:0/4148855278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c007040 con 0x7fa18c13db50 2023-07-29T13:28:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 1 -- 172.21.15.40:0/3533042190 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c03c8d0 con 0x7f85780b7360 2023-07-29T13:28:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 10 monclient: dump: 2023-07-29T13:28:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.821 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 1 -- 172.21.15.40:0/3533042190 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c03ca50 con 0x7f85780b7360 2023-07-29T13:28:16.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f8577fff640 5 monclient: authenticate success, global_id 4303 2023-07-29T13:28:16.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f8577fff640 10 monclient: _renew_subs 2023-07-29T13:28:16.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f8577fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:16.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f8577fff640 1 -- 172.21.15.40:0/3533042190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85781d76a0 con 0x7f85780b7360 2023-07-29T13:28:16.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 1 -- 172.21.15.40:0/3533042190 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c03ce00 con 0x7f85780b7360 2023-07-29T13:28:16.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f855ffff640 10 monclient: dump: 2023-07-29T13:28:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.824 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.824 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.824 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.824 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.824 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f85757fa640 4 set_mon_vals no callback set 2023-07-29T13:28:16.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f8577fff640 10 client.4303.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f8577fff640 10 monclient: _renew_subs 2023-07-29T13:28:16.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f8577fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:16.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f8577fff640 1 -- 172.21.15.40:0/3533042190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857806d2b0 con 0x7f85780b7360 2023-07-29T13:28:16.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.654+0000 7f8577fff640 1 librados: init done 2023-07-29T13:28:16.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.655+0000 7f8577fff640 10 client.4303.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:16.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.655+0000 7f8577fff640 15 client.4303.objecter _session_command_op_assign -1 1 2023-07-29T13:28:16.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.655+0000 7f8577fff640 20 client.4303.objecter _get_session osd=-1 returning homeless 2023-07-29T13:28:16.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.655+0000 7f8577fff640 10 client.4303.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.655+0000 7f8577fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:16.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.655+0000 7f8577fff640 1 -- 172.21.15.40:0/3533042190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8578069be0 con 0x7f85780b7360 2023-07-29T13:28:16.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.659+0000 7f0881a58640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08740072b0 0x7f087402feb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f086c000c00 tx=0x7f086c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.660+0000 7f08727fc640 10 client.?.objecter ms_handle_connect 0x7f08740072b0 2023-07-29T13:28:16.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.660+0000 7f08727fc640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:16.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.660+0000 7fa190f38640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18c13db50 0x7fa18c1d9220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c410 tx=0x7fa18800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.660+0000 7fa181ffb640 10 client.?.objecter ms_handle_connect 0x7fa18c13db50 2023-07-29T13:28:16.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.660+0000 7fa181ffb640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:16.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 1 -- 172.21.15.40:0/762864763 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c004220 con 0x7f0c340072b0 2023-07-29T13:28:16.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:16.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 10 monclient: dump: 2023-07-29T13:28:16.829 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.829 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.829 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.829 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.829 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.829 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.829 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.829 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.830 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 1 -- 172.21.15.40:0/762864763 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c0043a0 con 0x7f0c340072b0 2023-07-29T13:28:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 1 -- 172.21.15.40:0/762864763 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c007a70 con 0x7f0c340072b0 2023-07-29T13:28:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c40dd8640 5 monclient: authenticate success, global_id 4241 2023-07-29T13:28:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c40dd8640 10 monclient: _renew_subs 2023-07-29T13:28:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c40dd8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c40dd8640 1 -- 172.21.15.40:0/762864763 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3415f4a0 con 0x7f0c340072b0 2023-07-29T13:28:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c38d99640 4 set_mon_vals no callback set 2023-07-29T13:28:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c2affd640 10 monclient: dump: 2023-07-29T13:28:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.833 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c40dd8640 10 client.4241.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c40dd8640 10 monclient: _renew_subs 2023-07-29T13:28:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c40dd8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c40dd8640 1 -- 172.21.15.40:0/762864763 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3415f930 con 0x7f0c340072b0 2023-07-29T13:28:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.662+0000 7f0c40dd8640 1 librados: init done 2023-07-29T13:28:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.663+0000 7f0c40dd8640 10 client.4241.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.663+0000 7f0c40dd8640 15 client.4241.objecter _session_command_op_assign -1 1 2023-07-29T13:28:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.663+0000 7f0c40dd8640 20 client.4241.objecter _get_session osd=-1 returning homeless 2023-07-29T13:28:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.663+0000 7f0c40dd8640 10 client.4241.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.663+0000 7f0c40dd8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.663+0000 7f0c40dd8640 1 -- 172.21.15.40:0/762864763 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c34030a80 con 0x7f0c340072b0 2023-07-29T13:28:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 1 -- 172.21.15.40:0/1622460967 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc003d80 con 0x7fe3040afce0 2023-07-29T13:28:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:16.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 10 monclient: dump: 2023-07-29T13:28:16.836 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.836 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.836 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.836 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.837 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 1 -- 172.21.15.40:0/1622460967 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2fc005070 con 0x7fe3040afce0 2023-07-29T13:28:16.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe30b05a640 5 monclient: authenticate success, global_id 4235 2023-07-29T13:28:16.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe30b05a640 10 monclient: _renew_subs 2023-07-29T13:28:16.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe30b05a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe30b05a640 1 -- 172.21.15.40:0/1622460967 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3040a56b0 con 0x7fe3040afce0 2023-07-29T13:28:16.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 1 -- 172.21.15.40:0/1622460967 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc005460 con 0x7fe3040afce0 2023-07-29T13:28:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe2faffd640 10 monclient: dump: 2023-07-29T13:28:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.840 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe308855640 4 set_mon_vals no callback set 2023-07-29T13:28:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe30b05a640 10 client.4235.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe30b05a640 10 monclient: _renew_subs 2023-07-29T13:28:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe30b05a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe30b05a640 1 -- 172.21.15.40:0/1622460967 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3040b0fc0 con 0x7fe3040afce0 2023-07-29T13:28:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.665+0000 7fe30b05a640 1 librados: init done 2023-07-29T13:28:16.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 1 -- 172.21.15.40:0/1599888459 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8858007870 con 0x7f88700afce0 2023-07-29T13:28:16.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:16.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 10 monclient: dump: 2023-07-29T13:28:16.842 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.842 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.842 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.843 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.843 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.843 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.843 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.843 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.843 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.843 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 1 -- 172.21.15.40:0/1599888459 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88580079d0 con 0x7f88700afce0 2023-07-29T13:28:16.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8874a4f640 5 monclient: authenticate success, global_id 4232 2023-07-29T13:28:16.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8874a4f640 10 monclient: _renew_subs 2023-07-29T13:28:16.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8874a4f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8874a4f640 1 -- 172.21.15.40:0/1599888459 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887013dea0 con 0x7f88700afce0 2023-07-29T13:28:16.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 1 -- 172.21.15.40:0/1599888459 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8858007d80 con 0x7f88700afce0 2023-07-29T13:28:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8874a4f640 10 client.4232.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8866ffd640 10 monclient: dump: 2023-07-29T13:28:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.846 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.846 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.846 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.846 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.846 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.846 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.846 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8874a4f640 10 monclient: _renew_subs 2023-07-29T13:28:16.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8874a4f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7fe30b05a640 10 client.4235.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:16.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8874a4f640 1 -- 172.21.15.40:0/1599888459 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88700a8220 con 0x7f88700afce0 2023-07-29T13:28:16.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f8874a4f640 1 librados: init done 2023-07-29T13:28:16.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7f886d59a640 4 set_mon_vals no callback set 2023-07-29T13:28:16.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7fe30b05a640 15 client.4235.objecter _session_command_op_assign -1 1 2023-07-29T13:28:16.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7fe30b05a640 20 client.4235.objecter _get_session osd=-1 returning homeless 2023-07-29T13:28:16.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7fe30b05a640 10 client.4235.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7fe30b05a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.666+0000 7fe30b05a640 1 -- 172.21.15.40:0/1622460967 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe3040ae240 con 0x7fe3040afce0 2023-07-29T13:28:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.667+0000 7f8874a4f640 10 client.4232.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.667+0000 7f8874a4f640 15 client.4232.objecter _session_command_op_assign -1 1 2023-07-29T13:28:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.667+0000 7f8874a4f640 20 client.4232.objecter _get_session osd=-1 returning homeless 2023-07-29T13:28:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.667+0000 7f8874a4f640 10 client.4232.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.667+0000 7f8874a4f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:16.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.667+0000 7f8874a4f640 1 -- 172.21.15.40:0/1599888459 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f887006f1d0 con 0x7f88700afce0 2023-07-29T13:28:16.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.684+0000 7fa834ff9640 1 -- 172.21.15.40:0/3317623425 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fa828003ca0 con 0x7fa8380ae810 2023-07-29T13:28:16.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.685+0000 7fa834ff9640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa82406bfb0 0x7fa82406e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.685+0000 7fa8377fe640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa82406bfb0 0x7fa82406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.685+0000 7fa8377fe640 10 monclient: get_auth_request con 0x7fa82406bfb0 auth_method 0 2023-07-29T13:28:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.686+0000 7fa8377fe640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa82406bfb0 0x7fa82406e460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa82c035000 tx=0x7fa82c02e000 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.686+0000 7fa834ff9640 10 client.4308.objecter ms_handle_connect 0x7fa82406bfb0 2023-07-29T13:28:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.693+0000 7f855ffff640 1 -- 172.21.15.40:0/3533042190 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f856c03a040 con 0x7f85780b7360 2023-07-29T13:28:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.694+0000 7f855ffff640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f856406bf90 0x7f856406e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.694+0000 7f8576ffd640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f856406bf90 0x7f856406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.694+0000 7f8576ffd640 10 monclient: get_auth_request con 0x7f856406bf90 auth_method 0 2023-07-29T13:28:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.694+0000 7f8576ffd640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f856406bf90 0x7f856406e440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8568009650 tx=0x7f8568006ba0 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.694+0000 7f855ffff640 10 client.4303.objecter ms_handle_connect 0x7f856406bf90 2023-07-29T13:28:16.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 1 -- 172.21.15.40:0/3317623425 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa82805b740 con 0x7fa8380ae810 2023-07-29T13:28:16.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 10 client.4308.objecter ms_dispatch 0x7fa838000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 3 client.4308.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 3 client.4308.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 10 client.4308.objecter checking command 1 2023-07-29T13:28:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] conn(0x7fa824072380 0x7fa824074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 20 client.4308.objecter _get_session s=0x7fa824071cb0 osd=7 3 2023-07-29T13:28:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 20 client.4308.objecter put_session s=0x7fa824071cb0 osd=7 3 2023-07-29T13:28:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 15 client.4308.objecter _session_command_op_remove -1 1 2023-07-29T13:28:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 20 client.4308.objecter _get_session s=0x7fa824071cb0 osd=7 3 2023-07-29T13:28:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 20 client.4308.objecter get_session s=0x7fa824071cb0 osd=7 3 2023-07-29T13:28:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 15 client.4308.objecter _session_command_op_assign 7 1 2023-07-29T13:28:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 20 client.4308.objecter put_session s=0x7fa824071cb0 osd=7 4 2023-07-29T13:28:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 10 client.4308.objecter _send_command 1 2023-07-29T13:28:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 1 -- 172.21.15.40:0/3317623425 --> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa824074e40 con 0x7fa824072380 2023-07-29T13:28:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 20 client.4308.objecter dump_active .. 0 homeless 2023-07-29T13:28:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 1 -- 172.21.15.40:0/3317623425 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa828001110 con 0x7fa8380ae810 2023-07-29T13:28:16.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa834ff9640 10 monclient: handle_get_version_reply finishing 1 version 24 2023-07-29T13:28:16.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.702+0000 7fa83cf77640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] conn(0x7fa824072380 0x7fa824074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa83cf77640 10 monclient: get_auth_request con 0x7fa824072380 auth_method 0 2023-07-29T13:28:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa181ffb640 1 -- 172.21.15.40:0/4148855278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1880035f0 con 0x7fa18c13db50 2023-07-29T13:28:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa181ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa181ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa181ffb640 10 monclient: dump: 2023-07-29T13:28:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.858 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa181ffb640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa181ffb640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa181ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa181ffb640 1 -- 172.21.15.40:0/4148855278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa188003770 con 0x7fa18c13db50 2023-07-29T13:28:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa19399d640 5 monclient: authenticate success, global_id 4314 2023-07-29T13:28:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa19399d640 10 monclient: _renew_subs 2023-07-29T13:28:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa19399d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.703+0000 7fa19399d640 1 -- 172.21.15.40:0/4148855278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c1d9920 con 0x7fa18c13db50 2023-07-29T13:28:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7fa181ffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7fa181ffb640 1 -- 172.21.15.40:0/4148855278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188003b20 con 0x7fa18c13db50 2023-07-29T13:28:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7fa181ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7fa181ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7fa181ffb640 10 monclient: dump: 2023-07-29T13:28:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.862 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7fa1837fe640 4 set_mon_vals no callback set 2023-07-29T13:28:16.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7fa19399d640 10 client.4314.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7fa19399d640 10 monclient: _renew_subs 2023-07-29T13:28:16.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7fa19399d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7fa19399d640 1 -- 172.21.15.40:0/4148855278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c071280 con 0x7fa18c13db50 2023-07-29T13:28:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7fa19399d640 1 librados: init done 2023-07-29T13:28:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7f08727fc640 1 -- 172.21.15.40:0/2431520686 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c03ac60 con 0x7f08740072b0 2023-07-29T13:28:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7f08727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7f08727fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7f08727fc640 10 monclient: dump: 2023-07-29T13:28:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.865 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7f08727fc640 10 monclient: _finish_auth 0 2023-07-29T13:28:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7f08727fc640 10 monclient: _check_auth_tickets 2023-07-29T13:28:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7f08727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7f08727fc640 1 -- 172.21.15.40:0/2431520686 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c03ade0 con 0x7f08740072b0 2023-07-29T13:28:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7f08727fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7f08727fc640 1 -- 172.21.15.40:0/2431520686 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c044630 con 0x7f08740072b0 2023-07-29T13:28:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.704+0000 7f0883cbc640 5 monclient: authenticate success, global_id 4311 2023-07-29T13:28:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0883cbc640 10 monclient: _renew_subs 2023-07-29T13:28:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0883cbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0883cbc640 1 -- 172.21.15.40:0/2431520686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08740c8de0 con 0x7f08740072b0 2023-07-29T13:28:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7fa19399d640 10 client.4314.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7fa19399d640 15 client.4314.objecter _session_command_op_assign -1 1 2023-07-29T13:28:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7fa19399d640 20 client.4314.objecter _get_session osd=-1 returning homeless 2023-07-29T13:28:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7fa19399d640 10 client.4314.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7fa19399d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7fa19399d640 1 -- 172.21.15.40:0/4148855278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa18c0a9fc0 con 0x7fa18c13db50 2023-07-29T13:28:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f08727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f08727fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f08727fc640 10 monclient: dump: 2023-07-29T13:28:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:16.869 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:16.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0883cbc640 10 client.4311.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0883cbc640 10 monclient: _renew_subs 2023-07-29T13:28:16.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0883cbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0883cbc640 1 -- 172.21.15.40:0/2431520686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08740c1280 con 0x7f08740072b0 2023-07-29T13:28:16.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0883cbc640 1 librados: init done 2023-07-29T13:28:16.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0873fff640 4 set_mon_vals no callback set 2023-07-29T13:28:16.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0883cbc640 10 client.4311.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:16.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0883cbc640 15 client.4311.objecter _session_command_op_assign -1 1 2023-07-29T13:28:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.705+0000 7f0883cbc640 20 client.4311.objecter _get_session osd=-1 returning homeless 2023-07-29T13:28:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.706+0000 7f0883cbc640 10 client.4311.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.706+0000 7f0883cbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.706+0000 7f0883cbc640 1 -- 172.21.15.40:0/2431520686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0874030a80 con 0x7f08740072b0 2023-07-29T13:28:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.706+0000 7fa83cf77640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] conn(0x7fa824072380 0x7fa824074790 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.706+0000 7fa834ff9640 10 client.4308.objecter ms_handle_connect 0x7fa824072380 2023-07-29T13:28:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.706+0000 7f05a17fa640 1 -- 172.21.15.40:0/4007714830 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f0594003c60 con 0x7f05a40a86c0 2023-07-29T13:28:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.707+0000 7f05a17fa640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f059006bfb0 0x7f059006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.707+0000 7f05a3fff640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f059006bfb0 0x7f059006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.707+0000 7f05a3fff640 10 monclient: get_auth_request con 0x7f059006bfb0 auth_method 0 2023-07-29T13:28:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.708+0000 7f05a3fff640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f059006bfb0 0x7f059006e460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0598007fd0 tx=0x7f0598009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.708+0000 7f05a17fa640 10 client.4300.objecter ms_handle_connect 0x7f059006bfb0 2023-07-29T13:28:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.710+0000 7fe2faffd640 1 -- 172.21.15.40:0/1622460967 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fe2fc0055c0 con 0x7fe3040afce0 2023-07-29T13:28:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7fe2faffd640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe2e806bfe0 0x7fe2e806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 1 -- 172.21.15.40:0/4007714830 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f05940010c0 con 0x7f05a40a86c0 2023-07-29T13:28:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 10 client.4300.objecter ms_dispatch 0x7f05a4000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 3 client.4300.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 3 client.4300.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 10 client.4300.objecter checking command 1 2023-07-29T13:28:16.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7fe30a058640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe2e806bfe0 0x7fe2e806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x7f0590072380 0x7f0590074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 20 client.4300.objecter _get_session s=0x7f0590071cb0 osd=2 3 2023-07-29T13:28:16.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 20 client.4300.objecter put_session s=0x7f0590071cb0 osd=2 3 2023-07-29T13:28:16.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 15 client.4300.objecter _session_command_op_remove -1 1 2023-07-29T13:28:16.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 20 client.4300.objecter _get_session s=0x7f0590071cb0 osd=2 3 2023-07-29T13:28:16.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 20 client.4300.objecter get_session s=0x7f0590071cb0 osd=2 3 2023-07-29T13:28:16.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 15 client.4300.objecter _session_command_op_assign 2 1 2023-07-29T13:28:16.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 20 client.4300.objecter put_session s=0x7f0590071cb0 osd=2 4 2023-07-29T13:28:16.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 10 client.4300.objecter _send_command 1 2023-07-29T13:28:16.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 1 -- 172.21.15.40:0/4007714830 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0590074e40 con 0x7f0590072380 2023-07-29T13:28:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a17fa640 20 client.4300.objecter dump_active .. 0 homeless 2023-07-29T13:28:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7fe30a058640 10 monclient: get_auth_request con 0x7fe2e806bfe0 auth_method 0 2023-07-29T13:28:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7fe2faffd640 10 client.4235.objecter ms_handle_connect 0x7fe2e806bfe0 2023-07-29T13:28:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7fe30a058640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe2e806bfe0 0x7fe2e806e490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4035000 tx=0x7fe2f4030000 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.711+0000 7f05a9002640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x7f0590072380 0x7f0590074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.712+0000 7f05a9002640 10 monclient: get_auth_request con 0x7f0590072380 auth_method 0 2023-07-29T13:28:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.714+0000 7f855ffff640 1 -- 172.21.15.40:0/3533042190 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f856c03a2f0 con 0x7f85780b7360 2023-07-29T13:28:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.714+0000 7f855ffff640 10 client.4303.objecter ms_dispatch 0x7f8578000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.714+0000 7f855ffff640 3 client.4303.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.714+0000 7f855ffff640 3 client.4303.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 10 client.4303.objecter checking command 1 2023-07-29T13:28:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] conn(0x7f8564072360 0x7f8564074770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 20 client.4303.objecter _get_session s=0x7f8564071c90 osd=0 3 2023-07-29T13:28:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 20 client.4303.objecter put_session s=0x7f8564071c90 osd=0 3 2023-07-29T13:28:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 15 client.4303.objecter _session_command_op_remove -1 1 2023-07-29T13:28:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 20 client.4303.objecter _get_session s=0x7f8564071c90 osd=0 3 2023-07-29T13:28:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 20 client.4303.objecter get_session s=0x7f8564071c90 osd=0 3 2023-07-29T13:28:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 15 client.4303.objecter _session_command_op_assign 0 1 2023-07-29T13:28:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 20 client.4303.objecter put_session s=0x7f8564071c90 osd=0 4 2023-07-29T13:28:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 10 client.4303.objecter _send_command 1 2023-07-29T13:28:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 1 -- 172.21.15.40:0/3533042190 --> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8564074e20 con 0x7f8564072360 2023-07-29T13:28:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 20 client.4303.objecter dump_active .. 0 homeless 2023-07-29T13:28:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 1 -- 172.21.15.40:0/1622460967 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe2fc05be30 con 0x7fe3040afce0 2023-07-29T13:28:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 10 client.4235.objecter ms_dispatch 0x7fe304000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 3 client.4235.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 3 client.4235.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 10 client.4235.objecter checking command 1 2023-07-29T13:28:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] conn(0x7fe2e80723b0 0x7fe2e80747c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 20 client.4235.objecter _get_session s=0x7fe2e8071ce0 osd=3 3 2023-07-29T13:28:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 20 client.4235.objecter put_session s=0x7fe2e8071ce0 osd=3 3 2023-07-29T13:28:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 15 client.4235.objecter _session_command_op_remove -1 1 2023-07-29T13:28:16.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 20 client.4235.objecter _get_session s=0x7fe2e8071ce0 osd=3 3 2023-07-29T13:28:16.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 20 client.4235.objecter get_session s=0x7fe2e8071ce0 osd=3 3 2023-07-29T13:28:16.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 15 client.4235.objecter _session_command_op_assign 3 1 2023-07-29T13:28:16.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 20 client.4235.objecter put_session s=0x7fe2e8071ce0 osd=3 4 2023-07-29T13:28:16.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 10 client.4235.objecter _send_command 1 2023-07-29T13:28:16.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 1 -- 172.21.15.40:0/1622460967 --> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2e8074e70 con 0x7fe2e80723b0 2023-07-29T13:28:16.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 20 client.4235.objecter dump_active .. 0 homeless 2023-07-29T13:28:16.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 1 -- 172.21.15.40:0/1622460967 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2fc001820 con 0x7fe3040afce0 2023-07-29T13:28:16.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe2faffd640 10 monclient: handle_get_version_reply finishing 1 version 24 2023-07-29T13:28:16.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 1 -- 172.21.15.40:0/3533042190 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f856c075c60 con 0x7f85780b7360 2023-07-29T13:28:16.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f855ffff640 10 monclient: handle_get_version_reply finishing 1 version 24 2023-07-29T13:28:16.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7f85777fe640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] conn(0x7f8564072360 0x7f8564074770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.715+0000 7fe30a859640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] conn(0x7fe2e80723b0 0x7fe2e80747c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.716+0000 7f05a17fa640 1 -- 172.21.15.40:0/4007714830 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0594053cf0 con 0x7f05a40a86c0 2023-07-29T13:28:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.716+0000 7f05a17fa640 10 monclient: handle_get_version_reply finishing 1 version 24 2023-07-29T13:28:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.716+0000 7f05a17fa640 10 client.4300.objecter ms_handle_connect 0x7f0590072380 2023-07-29T13:28:16.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.716+0000 7f05a9002640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x7f0590072380 0x7f0590074790 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.716+0000 7fe30a859640 10 monclient: get_auth_request con 0x7fe2e80723b0 auth_method 0 2023-07-29T13:28:16.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.716+0000 7f85777fe640 10 monclient: get_auth_request con 0x7f8564072360 auth_method 0 2023-07-29T13:28:16.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.720+0000 7fe2faffd640 10 client.4235.objecter ms_handle_connect 0x7fe2e80723b0 2023-07-29T13:28:16.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.720+0000 7fe30a859640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] conn(0x7fe2e80723b0 0x7fe2e80747c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.721+0000 7f85777fe640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] conn(0x7f8564072360 0x7f8564074770 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.721+0000 7f855ffff640 10 client.4303.objecter ms_handle_connect 0x7f8564072360 2023-07-29T13:28:16.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.733+0000 7f8866ffd640 1 -- 172.21.15.40:0/1599888459 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f885800f070 con 0x7f88700afce0 2023-07-29T13:28:16.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.734+0000 7f8866ffd640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f885006bfe0 0x7f885006e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.734+0000 7f8867fff640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f885006bfe0 0x7f885006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.734+0000 7f8867fff640 10 monclient: get_auth_request con 0x7f885006bfe0 auth_method 0 2023-07-29T13:28:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.735+0000 7f8866ffd640 10 client.4232.objecter ms_handle_connect 0x7f885006bfe0 2023-07-29T13:28:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.735+0000 7f8867fff640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f885006bfe0 0x7f885006e490 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8860036000 tx=0x7f8860030000 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 1 -- 172.21.15.40:0/1599888459 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f885805c440 con 0x7f88700afce0 2023-07-29T13:28:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 10 client.4232.objecter ms_dispatch 0x7f8870000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 3 client.4232.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 3 client.4232.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 10 client.4232.objecter checking command 1 2023-07-29T13:28:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] conn(0x7f88500723b0 0x7f88500747c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 20 client.4232.objecter _get_session s=0x7f8850071ce0 osd=1 3 2023-07-29T13:28:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 20 client.4232.objecter put_session s=0x7f8850071ce0 osd=1 3 2023-07-29T13:28:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 15 client.4232.objecter _session_command_op_remove -1 1 2023-07-29T13:28:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 20 client.4232.objecter _get_session s=0x7f8850071ce0 osd=1 3 2023-07-29T13:28:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 20 client.4232.objecter get_session s=0x7f8850071ce0 osd=1 3 2023-07-29T13:28:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 15 client.4232.objecter _session_command_op_assign 1 1 2023-07-29T13:28:16.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 20 client.4232.objecter put_session s=0x7f8850071ce0 osd=1 4 2023-07-29T13:28:16.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 10 client.4232.objecter _send_command 1 2023-07-29T13:28:16.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 1 -- 172.21.15.40:0/1599888459 --> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8850074e70 con 0x7f88500723b0 2023-07-29T13:28:16.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 20 client.4232.objecter dump_active .. 0 homeless 2023-07-29T13:28:16.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 1 -- 172.21.15.40:0/1599888459 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f885800a410 con 0x7f88700afce0 2023-07-29T13:28:16.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.738+0000 7f8866ffd640 10 monclient: handle_get_version_reply finishing 1 version 24 2023-07-29T13:28:16.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.739+0000 7f886ed9d640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] conn(0x7f88500723b0 0x7f88500747c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.739+0000 7f886ed9d640 10 monclient: get_auth_request con 0x7f88500723b0 auth_method 0 2023-07-29T13:28:16.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.743+0000 7f886ed9d640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] conn(0x7f88500723b0 0x7f88500747c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.743+0000 7f8866ffd640 10 client.4232.objecter ms_handle_connect 0x7f88500723b0 2023-07-29T13:28:16.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.745+0000 7fa181ffb640 1 -- 172.21.15.40:0/4148855278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fa188003c80 con 0x7fa18c13db50 2023-07-29T13:28:16.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.746+0000 7fa181ffb640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa17006bf60 0x7fa17006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.746+0000 7fa191739640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa17006bf60 0x7fa17006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.746+0000 7fa191739640 10 monclient: get_auth_request con 0x7fa17006bf60 auth_method 0 2023-07-29T13:28:16.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.747+0000 7fa181ffb640 10 client.4314.objecter ms_handle_connect 0x7fa17006bf60 2023-07-29T13:28:16.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.747+0000 7fa191739640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa17006bf60 0x7fa17006e410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa17c035000 tx=0x7fa17c031000 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.761+0000 7f0c2affd640 1 -- 172.21.15.40:0/762864763 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f0c2c007bd0 con 0x7f0c340072b0 2023-07-29T13:28:16.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.762+0000 7f0c2affd640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f0c1c06bf60 0x7f0c1c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.762+0000 7f0c39d9b640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f0c1c06bf60 0x7f0c1c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.762+0000 7f0c39d9b640 10 monclient: get_auth_request con 0x7f0c1c06bf60 auth_method 0 2023-07-29T13:28:16.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.763+0000 7f0c39d9b640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f0c1c06bf60 0x7f0c1c06e410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c30036000 tx=0x7f0c3003c040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.763+0000 7f0c2affd640 10 client.4241.objecter ms_handle_connect 0x7f0c1c06bf60 2023-07-29T13:28:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 1 -- 172.21.15.40:0/762864763 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0c2c05bec0 con 0x7f0c340072b0 2023-07-29T13:28:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 10 client.4241.objecter ms_dispatch 0x7f0c3c000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 3 client.4241.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 3 client.4241.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 10 client.4241.objecter checking command 1 2023-07-29T13:28:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] conn(0x7f0c1c072330 0x7f0c1c074740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 20 client.4241.objecter _get_session s=0x7f0c1c071c60 osd=4 3 2023-07-29T13:28:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 20 client.4241.objecter put_session s=0x7f0c1c071c60 osd=4 3 2023-07-29T13:28:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 15 client.4241.objecter _session_command_op_remove -1 1 2023-07-29T13:28:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 20 client.4241.objecter _get_session s=0x7f0c1c071c60 osd=4 3 2023-07-29T13:28:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 20 client.4241.objecter get_session s=0x7f0c1c071c60 osd=4 3 2023-07-29T13:28:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 15 client.4241.objecter _session_command_op_assign 4 1 2023-07-29T13:28:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 20 client.4241.objecter put_session s=0x7f0c1c071c60 osd=4 4 2023-07-29T13:28:16.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 10 client.4241.objecter _send_command 1 2023-07-29T13:28:16.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 1 -- 172.21.15.40:0/762864763 --> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c1c074df0 con 0x7f0c1c072330 2023-07-29T13:28:16.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 20 client.4241.objecter dump_active .. 0 homeless 2023-07-29T13:28:16.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 1 -- 172.21.15.40:0/762864763 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c2c007e80 con 0x7f0c340072b0 2023-07-29T13:28:16.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.768+0000 7f0c2affd640 10 monclient: handle_get_version_reply finishing 1 version 24 2023-07-29T13:28:16.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.769+0000 7f0c3ad9d640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] conn(0x7f0c1c072330 0x7f0c1c074740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.769+0000 7f0c3ad9d640 10 monclient: get_auth_request con 0x7f0c1c072330 auth_method 0 2023-07-29T13:28:16.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.771+0000 7f05a17fa640 1 -- 172.21.15.40:0/4007714830 <== osd.2 v2:172.21.15.40:6800/85572 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0590074e40 con 0x7f0590072380 2023-07-29T13:28:16.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.771+0000 7f05a17fa640 10 client.4300.objecter ms_dispatch 0x7f05a4000b90 command_reply(tid 1: 0 ) v1 2023-07-29T13:28:16.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.771+0000 7f05a17fa640 10 client.4300.objecter _finish_command 1 = system:0 2023-07-29T13:28:16.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.771+0000 7f05a17fa640 20 client.4300.objecter put_session s=0x7f0590071cb0 osd=2 4 2023-07-29T13:28:16.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.771+0000 7f05a17fa640 15 client.4300.objecter _session_command_op_remove 2 1 2023-07-29T13:28:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.773+0000 7f0c2affd640 10 client.4241.objecter ms_handle_connect 0x7f0c1c072330 2023-07-29T13:28:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.773+0000 7f0c3ad9d640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] conn(0x7f0c1c072330 0x7f0c1c074740 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.774+0000 7f08727fc640 1 -- 172.21.15.40:0/2431520686 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f086c02f050 con 0x7f08740072b0 2023-07-29T13:28:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.775+0000 7f08727fc640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f086006bfb0 0x7f086006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.775+0000 7f0881257640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f086006bfb0 0x7f086006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.775+0000 7f0881257640 10 monclient: get_auth_request con 0x7f086006bfb0 auth_method 0 2023-07-29T13:28:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.776+0000 7f08727fc640 10 client.4311.objecter ms_handle_connect 0x7f086006bfb0 2023-07-29T13:28:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.776+0000 7f0881257640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f086006bfb0 0x7f086006e460 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f08740332f0 tx=0x7f0878008040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.776+0000 7f855ffff640 1 -- 172.21.15.40:0/3533042190 <== osd.0 v2:172.21.15.40:6808/85570 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8564074e20 con 0x7f8564072360 2023-07-29T13:28:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.776+0000 7f855ffff640 10 client.4303.objecter ms_dispatch 0x7f8578000b90 command_reply(tid 1: 0 ) v1 2023-07-29T13:28:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.776+0000 7f855ffff640 10 client.4303.objecter _finish_command 1 = system:0 2023-07-29T13:28:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.776+0000 7f855ffff640 20 client.4303.objecter put_session s=0x7f8564071c90 osd=0 4 2023-07-29T13:28:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.776+0000 7f855ffff640 15 client.4303.objecter _session_command_op_remove 0 1 2023-07-29T13:28:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 1 -- 172.21.15.40:0/2431520686 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f086c07dcd0 con 0x7f08740072b0 2023-07-29T13:28:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 10 client.4311.objecter ms_dispatch 0x7f087c000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 3 client.4311.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 3 client.4311.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 10 client.4311.objecter checking command 1 2023-07-29T13:28:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] conn(0x7f0860072380 0x7f0860074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 20 client.4311.objecter _get_session s=0x7f0860071cb0 osd=5 3 2023-07-29T13:28:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 20 client.4311.objecter put_session s=0x7f0860071cb0 osd=5 3 2023-07-29T13:28:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 15 client.4311.objecter _session_command_op_remove -1 1 2023-07-29T13:28:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 20 client.4311.objecter _get_session s=0x7f0860071cb0 osd=5 3 2023-07-29T13:28:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 20 client.4311.objecter get_session s=0x7f0860071cb0 osd=5 3 2023-07-29T13:28:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 15 client.4311.objecter _session_command_op_assign 5 1 2023-07-29T13:28:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 20 client.4311.objecter put_session s=0x7f0860071cb0 osd=5 4 2023-07-29T13:28:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 10 client.4311.objecter _send_command 1 2023-07-29T13:28:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 1 -- 172.21.15.40:0/2431520686 --> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0860074e40 con 0x7f0860072380 2023-07-29T13:28:16.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7f08727fc640 20 client.4311.objecter dump_active .. 0 homeless 2023-07-29T13:28:16.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 1 -- 172.21.15.40:0/4148855278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa1880010c0 con 0x7fa18c13db50 2023-07-29T13:28:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 10 client.4314.objecter ms_dispatch 0x7fa18c000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 3 client.4314.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 3 client.4314.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 10 client.4314.objecter checking command 1 2023-07-29T13:28:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] conn(0x7fa170072330 0x7fa170074740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 20 client.4314.objecter _get_session s=0x7fa170071c60 osd=6 3 2023-07-29T13:28:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 20 client.4314.objecter put_session s=0x7fa170071c60 osd=6 3 2023-07-29T13:28:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 15 client.4314.objecter _session_command_op_remove -1 1 2023-07-29T13:28:16.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 20 client.4314.objecter _get_session s=0x7fa170071c60 osd=6 3 2023-07-29T13:28:16.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 20 client.4314.objecter get_session s=0x7fa170071c60 osd=6 3 2023-07-29T13:28:16.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 15 client.4314.objecter _session_command_op_assign 6 1 2023-07-29T13:28:16.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 20 client.4314.objecter put_session s=0x7fa170071c60 osd=6 4 2023-07-29T13:28:16.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 10 client.4314.objecter _send_command 1 2023-07-29T13:28:16.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 1 -- 172.21.15.40:0/4148855278 --> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa170074df0 con 0x7fa170072330 2023-07-29T13:28:16.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 20 client.4314.objecter dump_active .. 0 homeless 2023-07-29T13:28:16.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 1 -- 172.21.15.40:0/4148855278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa188053c70 con 0x7fa18c13db50 2023-07-29T13:28:16.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.783+0000 7fa181ffb640 10 monclient: handle_get_version_reply finishing 1 version 24 2023-07-29T13:28:16.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.784+0000 7fa191f3a640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] conn(0x7fa170072330 0x7fa170074740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.784+0000 7f0882259640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] conn(0x7f0860072380 0x7f0860074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:16.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.784+0000 7f0882259640 10 monclient: get_auth_request con 0x7f0860072380 auth_method 0 2023-07-29T13:28:16.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.784+0000 7fa191f3a640 10 monclient: get_auth_request con 0x7fa170072330 auth_method 0 2023-07-29T13:28:16.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.785+0000 7f08727fc640 1 -- 172.21.15.40:0/2431520686 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f086c076050 con 0x7f08740072b0 2023-07-29T13:28:16.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.785+0000 7f08727fc640 10 monclient: handle_get_version_reply finishing 1 version 24 2023-07-29T13:28:16.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.788+0000 7f0882259640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] conn(0x7f0860072380 0x7f0860074790 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.788+0000 7f08727fc640 10 client.4311.objecter ms_handle_connect 0x7f0860072380 2023-07-29T13:28:16.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.788+0000 7fa834ff9640 1 -- 172.21.15.40:0/3317623425 <== osd.7 v2:172.21.15.45:6802/85182 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa824074e40 con 0x7fa824072380 2023-07-29T13:28:16.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.788+0000 7fa834ff9640 10 client.4308.objecter ms_dispatch 0x7fa838000b90 command_reply(tid 1: 0 ) v1 2023-07-29T13:28:16.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.788+0000 7fa834ff9640 10 client.4308.objecter _finish_command 1 = system:0 2023-07-29T13:28:16.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.788+0000 7fa834ff9640 20 client.4308.objecter put_session s=0x7fa824071cb0 osd=7 4 2023-07-29T13:28:16.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.788+0000 7fa834ff9640 15 client.4308.objecter _session_command_op_remove 7 1 2023-07-29T13:28:16.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.788+0000 7fa181ffb640 10 client.4314.objecter ms_handle_connect 0x7fa170072330 2023-07-29T13:28:16.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.788+0000 7fa191f3a640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] conn(0x7fa170072330 0x7fa170074740 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:16.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.790+0000 7fe2faffd640 1 -- 172.21.15.40:0/1622460967 <== osd.3 v2:172.21.15.40:6824/85571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe2e8074e70 con 0x7fe2e80723b0 2023-07-29T13:28:16.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.790+0000 7fe2faffd640 10 client.4235.objecter ms_dispatch 0x7fe304000b90 command_reply(tid 1: 0 ) v1 2023-07-29T13:28:16.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.790+0000 7fe2faffd640 10 client.4235.objecter _finish_command 1 = system:0 2023-07-29T13:28:16.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.790+0000 7fe2faffd640 20 client.4235.objecter put_session s=0x7fe2e8071ce0 osd=3 4 2023-07-29T13:28:16.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.790+0000 7fe2faffd640 15 client.4235.objecter _session_command_op_remove 3 1 2023-07-29T13:28:16.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7f8866ffd640 1 -- 172.21.15.40:0/1599888459 <== osd.1 v2:172.21.15.40:6816/85573 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8850074e70 con 0x7f88500723b0 2023-07-29T13:28:16.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7f8866ffd640 10 client.4232.objecter ms_dispatch 0x7f8870000b90 command_reply(tid 1: 0 ) v1 2023-07-29T13:28:16.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7f8866ffd640 10 client.4232.objecter _finish_command 1 = system:0 2023-07-29T13:28:16.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7f8866ffd640 20 client.4232.objecter put_session s=0x7f8850071ce0 osd=1 4 2023-07-29T13:28:16.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7f8866ffd640 15 client.4232.objecter _session_command_op_remove 1 1 2023-07-29T13:28:16.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7fa83e9da640 10 client.4308.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-07-29T13:28:16.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7fa83e9da640 15 client.4308.objecter _session_command_op_assign -1 2 2023-07-29T13:28:16.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7fa83e9da640 20 client.4308.objecter _get_session s=0x7fa824071cb0 osd=7 3 2023-07-29T13:28:16.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7fa83e9da640 20 client.4308.objecter put_session s=0x7fa824071cb0 osd=7 3 2023-07-29T13:28:16.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7fa83e9da640 20 client.4308.objecter _get_session s=0x7fa824071cb0 osd=7 3 2023-07-29T13:28:16.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7fa83e9da640 15 client.4308.objecter _session_command_op_remove -1 2 2023-07-29T13:28:16.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7fa83e9da640 20 client.4308.objecter get_session s=0x7fa824071cb0 osd=7 3 2023-07-29T13:28:16.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7fa83e9da640 15 client.4308.objecter _session_command_op_assign 7 2 2023-07-29T13:28:16.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7fa83e9da640 20 client.4308.objecter put_session s=0x7fa824071cb0 osd=7 4 2023-07-29T13:28:16.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7fa83e9da640 10 client.4308.objecter _send_command 2 2023-07-29T13:28:16.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.802+0000 7fa83e9da640 1 -- 172.21.15.40:0/3317623425 --> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa83806e490 con 0x7fa824072380 2023-07-29T13:28:16.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.803+0000 7f05a9803640 10 client.4300.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-07-29T13:28:16.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.803+0000 7f05a9803640 15 client.4300.objecter _session_command_op_assign -1 2 2023-07-29T13:28:16.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.803+0000 7f05a9803640 20 client.4300.objecter _get_session s=0x7f0590071cb0 osd=2 3 2023-07-29T13:28:16.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.803+0000 7f05a9803640 20 client.4300.objecter put_session s=0x7f0590071cb0 osd=2 3 2023-07-29T13:28:16.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.803+0000 7f05a9803640 20 client.4300.objecter _get_session s=0x7f0590071cb0 osd=2 3 2023-07-29T13:28:16.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.803+0000 7f05a9803640 15 client.4300.objecter _session_command_op_remove -1 2 2023-07-29T13:28:16.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.803+0000 7f05a9803640 20 client.4300.objecter get_session s=0x7f0590071cb0 osd=2 3 2023-07-29T13:28:16.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.803+0000 7f05a9803640 15 client.4300.objecter _session_command_op_assign 2 2 2023-07-29T13:28:16.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.803+0000 7f05a9803640 20 client.4300.objecter put_session s=0x7f0590071cb0 osd=2 4 2023-07-29T13:28:16.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.803+0000 7f05a9803640 10 client.4300.objecter _send_command 2 2023-07-29T13:28:16.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.803+0000 7f05a9803640 1 -- 172.21.15.40:0/4007714830 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f05a40aa930 con 0x7f0590072380 2023-07-29T13:28:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.806+0000 7f8577fff640 10 client.4303.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-07-29T13:28:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.806+0000 7f8577fff640 15 client.4303.objecter _session_command_op_assign -1 2 2023-07-29T13:28:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.806+0000 7f8577fff640 20 client.4303.objecter _get_session s=0x7f8564071c90 osd=0 3 2023-07-29T13:28:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.806+0000 7f8577fff640 20 client.4303.objecter put_session s=0x7f8564071c90 osd=0 3 2023-07-29T13:28:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.806+0000 7f8577fff640 20 client.4303.objecter _get_session s=0x7f8564071c90 osd=0 3 2023-07-29T13:28:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.806+0000 7f8577fff640 15 client.4303.objecter _session_command_op_remove -1 2 2023-07-29T13:28:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.806+0000 7f8577fff640 20 client.4303.objecter get_session s=0x7f8564071c90 osd=0 3 2023-07-29T13:28:16.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.806+0000 7f8577fff640 15 client.4303.objecter _session_command_op_assign 0 2 2023-07-29T13:28:16.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.806+0000 7f8577fff640 20 client.4303.objecter put_session s=0x7f8564071c90 osd=0 4 2023-07-29T13:28:16.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.806+0000 7f8577fff640 10 client.4303.objecter _send_command 2 2023-07-29T13:28:16.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.806+0000 7f8577fff640 1 -- 172.21.15.40:0/3533042190 --> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f857809cc80 con 0x7f8564072360 2023-07-29T13:28:16.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.810+0000 7fe30b05a640 10 client.4235.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-07-29T13:28:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.810+0000 7fe30b05a640 15 client.4235.objecter _session_command_op_assign -1 2 2023-07-29T13:28:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.810+0000 7fe30b05a640 20 client.4235.objecter _get_session s=0x7fe2e8071ce0 osd=3 3 2023-07-29T13:28:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.810+0000 7fe30b05a640 20 client.4235.objecter put_session s=0x7fe2e8071ce0 osd=3 3 2023-07-29T13:28:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.810+0000 7fe30b05a640 20 client.4235.objecter _get_session s=0x7fe2e8071ce0 osd=3 3 2023-07-29T13:28:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.810+0000 7fe30b05a640 15 client.4235.objecter _session_command_op_remove -1 2 2023-07-29T13:28:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.810+0000 7fe30b05a640 20 client.4235.objecter get_session s=0x7fe2e8071ce0 osd=3 3 2023-07-29T13:28:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.810+0000 7fe30b05a640 15 client.4235.objecter _session_command_op_assign 3 2 2023-07-29T13:28:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.810+0000 7fe30b05a640 20 client.4235.objecter put_session s=0x7fe2e8071ce0 osd=3 4 2023-07-29T13:28:16.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.810+0000 7fe30b05a640 10 client.4235.objecter _send_command 2 2023-07-29T13:28:16.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.810+0000 7fe30b05a640 1 -- 172.21.15.40:0/1622460967 --> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe3040ae840 con 0x7fe2e80723b0 2023-07-29T13:28:16.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a17fa640 1 -- 172.21.15.40:0/4007714830 <== osd.2 v2:172.21.15.40:6800/85572 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f059c00a3b0 con 0x7f0590072380 2023-07-29T13:28:16.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a17fa640 10 client.4300.objecter ms_dispatch 0x7f05a4000b90 command_reply(tid 2: 0 ) v1 2023-07-29T13:28:16.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a17fa640 10 client.4300.objecter _finish_command 2 = system:0 2023-07-29T13:28:16.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a17fa640 20 client.4300.objecter put_session s=0x7f0590071cb0 osd=2 4 2023-07-29T13:28:16.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a17fa640 15 client.4300.objecter _session_command_op_remove 2 2 2023-07-29T13:28:16.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 10 librados: watch_flush enter 2023-07-29T13:28:16.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 10 librados: watch_flush exit 2023-07-29T13:28:16.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 10 client.4300.objecter close_session for osd.2 2023-07-29T13:28:16.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 1 -- 172.21.15.40:0/4007714830 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x7f0590072380 msgr2=0x7f0590074790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x7f0590072380 0x7f0590074790 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 20 client.4300.objecter put_session s=0x7f0590071cb0 osd=2 1 2023-07-29T13:28:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 20 client.4300.objecter shutdown clearing up homeless session... 2023-07-29T13:28:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 10 client.4300.objecter successfully canceled tick 2023-07-29T13:28:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 1 -- 172.21.15.40:0/4007714830 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f059006bfb0 msgr2=0x7f059006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f059006bfb0 0x7f059006e460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0598007fd0 tx=0x7f0598009040 comp rx=0 tx=0).stop 2023-07-29T13:28:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 10 monclient: shutdown 2023-07-29T13:28:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 1 -- 172.21.15.40:0/4007714830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40a86c0 msgr2=0x7f05a41d6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9803640 1 --2- 172.21.15.40:0/4007714830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40a86c0 0x7f05a41d6960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f059400c410 tx=0x7f059400c8d0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.814+0000 7f05a9002640 1 -- 172.21.15.40:0/4007714830 reap_dead start 2023-07-29T13:28:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.815+0000 7f05a9803640 1 -- 172.21.15.40:0/4007714830 shutdown_connections 2023-07-29T13:28:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.815+0000 7f05a9803640 1 -- 172.21.15.40:0/4007714830 >> 172.21.15.40:0/4007714830 conn(0x7f05a406a2b0 msgr2=0x7f05a40adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.815+0000 7f05a9803640 1 -- 172.21.15.40:0/4007714830 shutdown_connections 2023-07-29T13:28:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.815+0000 7f05a9803640 1 -- 172.21.15.40:0/4007714830 wait complete. 2023-07-29T13:28:16.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.815+0000 7f05a9803640 1 librados: shutdown 2023-07-29T13:28:16.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.817+0000 7f855ffff640 1 -- 172.21.15.40:0/3533042190 <== osd.0 v2:172.21.15.40:6808/85570 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f857809cc80 con 0x7f8564072360 2023-07-29T13:28:16.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.817+0000 7f855ffff640 10 client.4303.objecter ms_dispatch 0x7f8578000b90 command_reply(tid 2: 0 ) v1 2023-07-29T13:28:16.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.817+0000 7f855ffff640 10 client.4303.objecter _finish_command 2 = system:0 2023-07-29T13:28:16.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.817+0000 7f855ffff640 20 client.4303.objecter put_session s=0x7f8564071c90 osd=0 4 2023-07-29T13:28:16.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.817+0000 7f855ffff640 15 client.4303.objecter _session_command_op_remove 0 2 2023-07-29T13:28:16.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa834ff9640 1 -- 172.21.15.40:0/3317623425 <== osd.7 v2:172.21.15.45:6802/85182 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa83806e490 con 0x7fa824072380 2023-07-29T13:28:16.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa834ff9640 10 client.4308.objecter ms_dispatch 0x7fa838000b90 command_reply(tid 2: 0 ) v1 2023-07-29T13:28:16.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa834ff9640 10 client.4308.objecter _finish_command 2 = system:0 2023-07-29T13:28:16.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa834ff9640 20 client.4308.objecter put_session s=0x7fa824071cb0 osd=7 4 2023-07-29T13:28:16.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa834ff9640 15 client.4308.objecter _session_command_op_remove 7 2 2023-07-29T13:28:16.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 10 librados: watch_flush enter 2023-07-29T13:28:16.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 10 librados: watch_flush exit 2023-07-29T13:28:16.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 10 client.4303.objecter close_session for osd.0 2023-07-29T13:28:16.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 1 -- 172.21.15.40:0/3533042190 >> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] conn(0x7f8564072360 msgr2=0x7f8564074770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] conn(0x7f8564072360 0x7f8564074770 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 20 client.4303.objecter put_session s=0x7f8564071c90 osd=0 1 2023-07-29T13:28:16.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 20 client.4303.objecter shutdown clearing up homeless session... 2023-07-29T13:28:16.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 10 client.4303.objecter successfully canceled tick 2023-07-29T13:28:16.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 1 -- 172.21.15.40:0/3533042190 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f856406bf90 msgr2=0x7f856406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f856406bf90 0x7f856406e440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8568009650 tx=0x7f8568006ba0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 10 monclient: shutdown 2023-07-29T13:28:16.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 1 -- 172.21.15.40:0/3533042190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85780b7360 msgr2=0x7f857813d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 1 --2- 172.21.15.40:0/3533042190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85780b7360 0x7f857813d790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f856c009890 tx=0x7f856c0046c0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 10 librados: watch_flush enter 2023-07-29T13:28:16.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 10 librados: watch_flush exit 2023-07-29T13:28:16.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 10 client.4308.objecter close_session for osd.7 2023-07-29T13:28:16.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 1 -- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] conn(0x7fa824072380 msgr2=0x7fa824074790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] conn(0x7fa824072380 0x7fa824074790 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 20 client.4308.objecter put_session s=0x7fa824071cb0 osd=7 1 2023-07-29T13:28:16.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 20 client.4308.objecter shutdown clearing up homeless session... 2023-07-29T13:28:16.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 10 client.4308.objecter successfully canceled tick 2023-07-29T13:28:16.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 1 -- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa82406bfb0 msgr2=0x7fa82406e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa82406bfb0 0x7fa82406e460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa82c035000 tx=0x7fa82c02e000 comp rx=0 tx=0).stop 2023-07-29T13:28:16.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f85777fe640 1 -- 172.21.15.40:0/3533042190 reap_dead start 2023-07-29T13:28:16.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 10 monclient: shutdown 2023-07-29T13:28:16.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 1 -- 172.21.15.40:0/3533042190 shutdown_connections 2023-07-29T13:28:16.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7f8577fff640 1 -- 172.21.15.40:0/3533042190 >> 172.21.15.40:0/3533042190 conn(0x7f857806a2b0 msgr2=0x7f85780b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.818+0000 7fa83e9da640 1 -- 172.21.15.40:0/3317623425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8380ae810 msgr2=0x7fa8380a59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.819+0000 7fa83e9da640 1 --2- 172.21.15.40:0/3317623425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8380ae810 0x7fa8380a59d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa82800c410 tx=0x7fa82800c8d0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.819+0000 7fa83cf77640 1 -- 172.21.15.40:0/3317623425 reap_dead start 2023-07-29T13:28:16.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.819+0000 7f8577fff640 1 -- 172.21.15.40:0/3533042190 shutdown_connections 2023-07-29T13:28:16.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.819+0000 7f8577fff640 1 -- 172.21.15.40:0/3533042190 wait complete. 2023-07-29T13:28:16.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.819+0000 7f8577fff640 1 librados: shutdown 2023-07-29T13:28:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.819+0000 7fa83e9da640 1 -- 172.21.15.40:0/3317623425 shutdown_connections 2023-07-29T13:28:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.819+0000 7fa83e9da640 1 -- 172.21.15.40:0/3317623425 >> 172.21.15.40:0/3317623425 conn(0x7fa83806a2b0 msgr2=0x7fa8380ad900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.819+0000 7fa83e9da640 1 -- 172.21.15.40:0/3317623425 shutdown_connections 2023-07-29T13:28:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.819+0000 7fa83e9da640 1 -- 172.21.15.40:0/3317623425 wait complete. 2023-07-29T13:28:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7fa83e9da640 1 librados: shutdown 2023-07-29T13:28:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7f8874a4f640 10 client.4232.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-07-29T13:28:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7f8874a4f640 15 client.4232.objecter _session_command_op_assign -1 2 2023-07-29T13:28:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7f8874a4f640 20 client.4232.objecter _get_session s=0x7f8850071ce0 osd=1 3 2023-07-29T13:28:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7f8874a4f640 20 client.4232.objecter put_session s=0x7f8850071ce0 osd=1 3 2023-07-29T13:28:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7f8874a4f640 20 client.4232.objecter _get_session s=0x7f8850071ce0 osd=1 3 2023-07-29T13:28:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7f8874a4f640 15 client.4232.objecter _session_command_op_remove -1 2 2023-07-29T13:28:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7f8874a4f640 20 client.4232.objecter get_session s=0x7f8850071ce0 osd=1 3 2023-07-29T13:28:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7f8874a4f640 15 client.4232.objecter _session_command_op_assign 1 2 2023-07-29T13:28:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7f8874a4f640 20 client.4232.objecter put_session s=0x7f8850071ce0 osd=1 4 2023-07-29T13:28:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7f8874a4f640 10 client.4232.objecter _send_command 2 2023-07-29T13:28:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.820+0000 7f8874a4f640 1 -- 172.21.15.40:0/1599888459 --> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f887013dd20 con 0x7f88500723b0 2023-07-29T13:28:16.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.822+0000 7fe2faffd640 1 -- 172.21.15.40:0/1622460967 <== osd.3 v2:172.21.15.40:6824/85571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe30000a2f0 con 0x7fe2e80723b0 2023-07-29T13:28:16.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.822+0000 7fe2faffd640 10 client.4235.objecter ms_dispatch 0x7fe304000b90 command_reply(tid 2: 0 ) v1 2023-07-29T13:28:16.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.822+0000 7fe2faffd640 10 client.4235.objecter _finish_command 2 = system:0 2023-07-29T13:28:16.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.822+0000 7fe2faffd640 20 client.4235.objecter put_session s=0x7fe2e8071ce0 osd=3 4 2023-07-29T13:28:16.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.822+0000 7fe2faffd640 15 client.4235.objecter _session_command_op_remove 3 2 2023-07-29T13:28:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.823+0000 7fe30b05a640 10 librados: watch_flush enter 2023-07-29T13:28:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.823+0000 7fe30b05a640 10 librados: watch_flush exit 2023-07-29T13:28:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.823+0000 7fe30b05a640 10 client.4235.objecter close_session for osd.3 2023-07-29T13:28:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.823+0000 7fe30b05a640 1 -- 172.21.15.40:0/1622460967 >> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] conn(0x7fe2e80723b0 msgr2=0x7fe2e80747c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.823+0000 7fe30b05a640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] conn(0x7fe2e80723b0 0x7fe2e80747c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.823+0000 7fe30b05a640 20 client.4235.objecter put_session s=0x7fe2e8071ce0 osd=3 1 2023-07-29T13:28:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.823+0000 7fe30b05a640 20 client.4235.objecter shutdown clearing up homeless session... 2023-07-29T13:28:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.823+0000 7fe30b05a640 10 client.4235.objecter successfully canceled tick 2023-07-29T13:28:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.824+0000 7fe30b05a640 1 -- 172.21.15.40:0/1622460967 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe2e806bfe0 msgr2=0x7fe2e806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.824+0000 7fe30b05a640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe2e806bfe0 0x7fe2e806e490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4035000 tx=0x7fe2f4030000 comp rx=0 tx=0).stop 2023-07-29T13:28:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.824+0000 7fe30b05a640 10 monclient: shutdown 2023-07-29T13:28:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.824+0000 7fe30b05a640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.824+0000 7fe30b05a640 1 -- 172.21.15.40:0/1622460967 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe3040afce0 msgr2=0x7fe3040a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.824+0000 7fe30b05a640 1 --2- 172.21.15.40:0/1622460967 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe3040afce0 0x7fe3040a4e70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00c7f0 tx=0x7fe2fc00ccb0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.824+0000 7fe30a859640 1 -- 172.21.15.40:0/1622460967 reap_dead start 2023-07-29T13:28:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.824+0000 7fe30b05a640 1 -- 172.21.15.40:0/1622460967 shutdown_connections 2023-07-29T13:28:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.824+0000 7fe30b05a640 1 -- 172.21.15.40:0/1622460967 >> 172.21.15.40:0/1622460967 conn(0x7fe30406a2b0 msgr2=0x7fe3040a21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.825+0000 7fe30b05a640 1 -- 172.21.15.40:0/1622460967 shutdown_connections 2023-07-29T13:28:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.826+0000 7fe30b05a640 1 -- 172.21.15.40:0/1622460967 wait complete. 2023-07-29T13:28:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.826+0000 7fe30b05a640 1 librados: shutdown 2023-07-29T13:28:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.828+0000 7f8866ffd640 1 -- 172.21.15.40:0/1599888459 <== osd.1 v2:172.21.15.40:6816/85573 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8868009c90 con 0x7f88500723b0 2023-07-29T13:28:16.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.828+0000 7f8866ffd640 10 client.4232.objecter ms_dispatch 0x7f8870000b90 command_reply(tid 2: 0 ) v1 2023-07-29T13:28:16.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.828+0000 7f8866ffd640 10 client.4232.objecter _finish_command 2 = system:0 2023-07-29T13:28:16.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.828+0000 7f8866ffd640 20 client.4232.objecter put_session s=0x7f8850071ce0 osd=1 4 2023-07-29T13:28:16.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.828+0000 7f8866ffd640 15 client.4232.objecter _session_command_op_remove 1 2 2023-07-29T13:28:16.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 10 librados: watch_flush enter 2023-07-29T13:28:16.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 10 librados: watch_flush exit 2023-07-29T13:28:16.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 10 client.4232.objecter close_session for osd.1 2023-07-29T13:28:16.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 1 -- 172.21.15.40:0/1599888459 >> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] conn(0x7f88500723b0 msgr2=0x7f88500747c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] conn(0x7f88500723b0 0x7f88500747c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 20 client.4232.objecter put_session s=0x7f8850071ce0 osd=1 1 2023-07-29T13:28:16.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 20 client.4232.objecter shutdown clearing up homeless session... 2023-07-29T13:28:16.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 10 client.4232.objecter successfully canceled tick 2023-07-29T13:28:16.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 1 -- 172.21.15.40:0/1599888459 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f885006bfe0 msgr2=0x7f885006e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f885006bfe0 0x7f885006e490 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8860036000 tx=0x7f8860030000 comp rx=0 tx=0).stop 2023-07-29T13:28:16.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 10 monclient: shutdown 2023-07-29T13:28:16.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 1 -- 172.21.15.40:0/1599888459 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f88700afce0 msgr2=0x7f88700a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.829+0000 7f8874a4f640 1 --2- 172.21.15.40:0/1599888459 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f88700afce0 0x7f88700a5090 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f885800c7f0 tx=0x7f885800cbf0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.946 INFO:teuthology.orchestra.run.smithi040.stdout:73014444037 2023-07-29T13:28:16.947 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-29T13:28:16.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.830+0000 7f886ed9d640 1 -- 172.21.15.40:0/1599888459 reap_dead start 2023-07-29T13:28:16.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.830+0000 7f8874a4f640 1 -- 172.21.15.40:0/1599888459 shutdown_connections 2023-07-29T13:28:16.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.830+0000 7f8874a4f640 1 -- 172.21.15.40:0/1599888459 >> 172.21.15.40:0/1599888459 conn(0x7f887006a2b0 msgr2=0x7f88700a8760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.830+0000 7f8874a4f640 1 -- 172.21.15.40:0/1599888459 shutdown_connections 2023-07-29T13:28:16.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.830+0000 7f8874a4f640 1 -- 172.21.15.40:0/1599888459 wait complete. 2023-07-29T13:28:16.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.830+0000 7f8874a4f640 1 librados: shutdown 2023-07-29T13:28:16.949 INFO:teuthology.orchestra.run.smithi040.stdout:73014444038 2023-07-29T13:28:16.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-29T13:28:16.950 INFO:teuthology.orchestra.run.smithi040.stdout:60129542150 2023-07-29T13:28:16.950 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-29T13:28:16.951 INFO:teuthology.orchestra.run.smithi040.stdout:73014444038 2023-07-29T13:28:16.951 INFO:teuthology.orchestra.run.smithi040.stdout:73014444037 2023-07-29T13:28:16.952 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-29T13:28:16.952 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-29T13:28:16.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.853+0000 7f08727fc640 1 -- 172.21.15.40:0/2431520686 <== osd.5 v2:172.21.15.45:6812/85181 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0860074e40 con 0x7f0860072380 2023-07-29T13:28:16.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.853+0000 7f08727fc640 10 client.4311.objecter ms_dispatch 0x7f087c000b90 command_reply(tid 1: 0 ) v1 2023-07-29T13:28:16.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.853+0000 7f08727fc640 10 client.4311.objecter _finish_command 1 = system:0 2023-07-29T13:28:16.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.853+0000 7f08727fc640 20 client.4311.objecter put_session s=0x7f0860071cb0 osd=5 4 2023-07-29T13:28:16.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.853+0000 7f08727fc640 15 client.4311.objecter _session_command_op_remove 5 1 2023-07-29T13:28:16.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.867+0000 7f0883cbc640 10 client.4311.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-07-29T13:28:16.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.867+0000 7f0883cbc640 15 client.4311.objecter _session_command_op_assign -1 2 2023-07-29T13:28:16.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.867+0000 7f0883cbc640 20 client.4311.objecter _get_session s=0x7f0860071cb0 osd=5 3 2023-07-29T13:28:16.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.867+0000 7f0883cbc640 20 client.4311.objecter put_session s=0x7f0860071cb0 osd=5 3 2023-07-29T13:28:16.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.867+0000 7f0883cbc640 20 client.4311.objecter _get_session s=0x7f0860071cb0 osd=5 3 2023-07-29T13:28:16.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.867+0000 7f0883cbc640 15 client.4311.objecter _session_command_op_remove -1 2 2023-07-29T13:28:16.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.867+0000 7f0883cbc640 20 client.4311.objecter get_session s=0x7f0860071cb0 osd=5 3 2023-07-29T13:28:16.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.867+0000 7f0883cbc640 15 client.4311.objecter _session_command_op_assign 5 2 2023-07-29T13:28:16.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.867+0000 7f0883cbc640 20 client.4311.objecter put_session s=0x7f0860071cb0 osd=5 4 2023-07-29T13:28:16.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.867+0000 7f0883cbc640 10 client.4311.objecter _send_command 2 2023-07-29T13:28:16.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.867+0000 7f0883cbc640 1 -- 172.21.15.40:0/2431520686 --> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0874036b50 con 0x7f0860072380 2023-07-29T13:28:16.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f08727fc640 1 -- 172.21.15.40:0/2431520686 <== osd.5 v2:172.21.15.45:6812/85181 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f087c066990 con 0x7f0860072380 2023-07-29T13:28:16.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f08727fc640 10 client.4311.objecter ms_dispatch 0x7f087c000b90 command_reply(tid 2: 0 ) v1 2023-07-29T13:28:16.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f08727fc640 10 client.4311.objecter _finish_command 2 = system:0 2023-07-29T13:28:16.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f08727fc640 20 client.4311.objecter put_session s=0x7f0860071cb0 osd=5 4 2023-07-29T13:28:16.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f08727fc640 15 client.4311.objecter _session_command_op_remove 5 2 2023-07-29T13:28:16.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 10 librados: watch_flush enter 2023-07-29T13:28:16.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 10 librados: watch_flush exit 2023-07-29T13:28:16.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 10 client.4311.objecter close_session for osd.5 2023-07-29T13:28:16.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 1 -- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] conn(0x7f0860072380 msgr2=0x7f0860074790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] conn(0x7f0860072380 0x7f0860074790 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 20 client.4311.objecter put_session s=0x7f0860071cb0 osd=5 1 2023-07-29T13:28:16.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 20 client.4311.objecter shutdown clearing up homeless session... 2023-07-29T13:28:16.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 10 client.4311.objecter successfully canceled tick 2023-07-29T13:28:16.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 1 -- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f086006bfb0 msgr2=0x7f086006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f086006bfb0 0x7f086006e460 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f08740332f0 tx=0x7f0878008040 comp rx=0 tx=0).stop 2023-07-29T13:28:16.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 10 monclient: shutdown 2023-07-29T13:28:16.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 1 -- 172.21.15.40:0/2431520686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08740072b0 msgr2=0x7f087402feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 1 --2- 172.21.15.40:0/2431520686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08740072b0 0x7f087402feb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f086c000c00 tx=0x7f086c0046c0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0882259640 1 -- 172.21.15.40:0/2431520686 reap_dead start 2023-07-29T13:28:16.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 1 -- 172.21.15.40:0/2431520686 shutdown_connections 2023-07-29T13:28:16.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 1 -- 172.21.15.40:0/2431520686 >> 172.21.15.40:0/2431520686 conn(0x7f0874039e90 msgr2=0x7f08740c2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 1 -- 172.21.15.40:0/2431520686 shutdown_connections 2023-07-29T13:28:16.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 1 -- 172.21.15.40:0/2431520686 wait complete. 2023-07-29T13:28:16.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.879+0000 7f0883cbc640 1 librados: shutdown 2023-07-29T13:28:16.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.880+0000 7fa181ffb640 1 -- 172.21.15.40:0/4148855278 <== osd.6 v2:172.21.15.45:6810/85184 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa170074df0 con 0x7fa170072330 2023-07-29T13:28:16.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.880+0000 7fa181ffb640 10 client.4314.objecter ms_dispatch 0x7fa18c000b90 command_reply(tid 1: 0 ) v1 2023-07-29T13:28:16.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.880+0000 7fa181ffb640 10 client.4314.objecter _finish_command 1 = system:0 2023-07-29T13:28:16.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.880+0000 7fa181ffb640 20 client.4314.objecter put_session s=0x7fa170071c60 osd=6 4 2023-07-29T13:28:16.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.880+0000 7fa181ffb640 15 client.4314.objecter _session_command_op_remove 6 1 2023-07-29T13:28:16.962 INFO:teuthology.orchestra.run.smithi040.stdout:64424509446 2023-07-29T13:28:16.962 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-29T13:28:16.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.894+0000 7fa19399d640 10 client.4314.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-07-29T13:28:16.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.894+0000 7fa19399d640 15 client.4314.objecter _session_command_op_assign -1 2 2023-07-29T13:28:16.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.894+0000 7fa19399d640 20 client.4314.objecter _get_session s=0x7fa170071c60 osd=6 3 2023-07-29T13:28:16.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.894+0000 7fa19399d640 20 client.4314.objecter put_session s=0x7fa170071c60 osd=6 3 2023-07-29T13:28:16.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.894+0000 7fa19399d640 20 client.4314.objecter _get_session s=0x7fa170071c60 osd=6 3 2023-07-29T13:28:16.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.894+0000 7fa19399d640 15 client.4314.objecter _session_command_op_remove -1 2 2023-07-29T13:28:16.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.894+0000 7fa19399d640 20 client.4314.objecter get_session s=0x7fa170071c60 osd=6 3 2023-07-29T13:28:16.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.894+0000 7fa19399d640 15 client.4314.objecter _session_command_op_assign 6 2 2023-07-29T13:28:16.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.894+0000 7fa19399d640 20 client.4314.objecter put_session s=0x7fa170071c60 osd=6 4 2023-07-29T13:28:16.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.894+0000 7fa19399d640 10 client.4314.objecter _send_command 2 2023-07-29T13:28:16.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.894+0000 7fa19399d640 1 -- 172.21.15.40:0/4148855278 --> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa18c140d40 con 0x7fa170072330 2023-07-29T13:28:16.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.905+0000 7fa181ffb640 1 -- 172.21.15.40:0/4148855278 <== osd.6 v2:172.21.15.45:6810/85184 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa18400f040 con 0x7fa170072330 2023-07-29T13:28:16.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.905+0000 7fa181ffb640 10 client.4314.objecter ms_dispatch 0x7fa18c000b90 command_reply(tid 2: 0 ) v1 2023-07-29T13:28:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.905+0000 7fa181ffb640 10 client.4314.objecter _finish_command 2 = system:0 2023-07-29T13:28:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.905+0000 7fa181ffb640 20 client.4314.objecter put_session s=0x7fa170071c60 osd=6 4 2023-07-29T13:28:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.905+0000 7fa181ffb640 15 client.4314.objecter _session_command_op_remove 6 2 2023-07-29T13:28:16.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 10 librados: watch_flush enter 2023-07-29T13:28:16.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 10 librados: watch_flush exit 2023-07-29T13:28:16.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 10 client.4314.objecter close_session for osd.6 2023-07-29T13:28:16.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 1 -- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] conn(0x7fa170072330 msgr2=0x7fa170074740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] conn(0x7fa170072330 0x7fa170074740 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 20 client.4314.objecter put_session s=0x7fa170071c60 osd=6 1 2023-07-29T13:28:16.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 20 client.4314.objecter shutdown clearing up homeless session... 2023-07-29T13:28:16.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 10 client.4314.objecter successfully canceled tick 2023-07-29T13:28:16.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 1 -- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa17006bf60 msgr2=0x7fa17006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa17006bf60 0x7fa17006e410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa17c035000 tx=0x7fa17c031000 comp rx=0 tx=0).stop 2023-07-29T13:28:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 10 monclient: shutdown 2023-07-29T13:28:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 1 -- 172.21.15.40:0/4148855278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18c13db50 msgr2=0x7fa18c1d9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 1 --2- 172.21.15.40:0/4148855278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18c13db50 0x7fa18c1d9220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c410 tx=0x7fa18800c8d0 comp rx=0 tx=0).stop 2023-07-29T13:28:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa191f3a640 1 -- 172.21.15.40:0/4148855278 reap_dead start 2023-07-29T13:28:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 1 -- 172.21.15.40:0/4148855278 shutdown_connections 2023-07-29T13:28:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 1 -- 172.21.15.40:0/4148855278 >> 172.21.15.40:0/4148855278 conn(0x7fa18c06a2b0 msgr2=0x7fa18c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:16.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 1 -- 172.21.15.40:0/4148855278 shutdown_connections 2023-07-29T13:28:16.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 1 -- 172.21.15.40:0/4148855278 wait complete. 2023-07-29T13:28:16.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:16.906+0000 7fa19399d640 1 librados: shutdown 2023-07-29T13:28:16.969 INFO:teuthology.orchestra.run.smithi040.stdout:64424509445 2023-07-29T13:28:16.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-29T13:28:17.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.141+0000 7fe2cfee3640 10 monclient: get_monmap_and_config 2023-07-29T13:28:17.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.141+0000 7fe2cfee3640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.141+0000 7fe2cfee3640 10 monclient: monmap: 2023-07-29T13:28:17.144 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.144 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.144 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:17.142921+0000 2023-07-29T13:28:17.144 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:17.142921+0000 2023-07-29T13:28:17.144 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:17.145 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.142+0000 7fe2cfee3640 1 Processor -- start 2023-07-29T13:28:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.142+0000 7fe2cfee3640 1 -- start start 2023-07-29T13:28:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.142+0000 7fe2cfee3640 10 monclient: init 2023-07-29T13:28:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.143+0000 7fe2cfee3640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.143+0000 7fe2cfee3640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2023-07-29T13:28:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.143+0000 7fe2cfee3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2c80b3ae0 0x7fe2c813dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.144+0000 7fe2cfee3640 10 monclient(hunting): picked mon.noname-a con 0x7fe2c80b3ae0 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.144+0000 7fe2cfee3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe2c80a8910 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.144+0000 7fe2cfee3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2c8141320 0x7fe2c813e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.144+0000 7fe2cfee3640 10 monclient(hunting): picked mon.noname-b con 0x7fe2c8141320 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.144+0000 7fe2cfee3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c80b3eb0 con 0x7fe2c80a8910 2023-07-29T13:28:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.144+0000 7fe2cfee3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.144+0000 7fe2cfee3640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c80b0f10 con 0x7fe2c80b3ae0 2023-07-29T13:28:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.144+0000 7fe2cfee3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.144+0000 7fe2cfee3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80a9950 con 0x7fe2c8141320 2023-07-29T13:28:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.144+0000 7fe2cfee3640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.144+0000 7fe2cfee3640 10 monclient(hunting): authenticate will time out at 1419.277167s 2023-07-29T13:28:17.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.145+0000 7fe2cd47e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe2c80a8910 0x7fe2c8140c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51278/0 (socket says 172.21.15.40:51278) 2023-07-29T13:28:17.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.145+0000 7fe2cd47e640 1 -- 172.21.15.40:0/3001920036 learned_addr learned my addr 172.21.15.40:0/3001920036 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:28:17.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.145+0000 7fe2ce480640 1 --2- 172.21.15.40:0/3001920036 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2c8141320 0x7fe2c813e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.145+0000 7fe2ce480640 10 monclient(hunting): get_auth_request con 0x7fe2c8141320 auth_method 0 2023-07-29T13:28:17.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.145+0000 7fe2ce480640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.145+0000 7fe2ce480640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.145+0000 7fe2ce480640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.147+0000 7fe2ce480640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.147+0000 7fe2ce480640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.147+0000 7fe2ce480640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.149+0000 7fe2ce480640 10 monclient(hunting): handle_auth_done global_id 4244 payload 274 2023-07-29T13:28:17.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.149+0000 7fe2ce480640 1 -- 172.21.15.40:0/3001920036 >> v2:172.21.15.40:3300/0 conn(0x7fe2c80b3ae0 msgr2=0x7fe2c813dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.149+0000 7fe2ce480640 1 --2- 172.21.15.40:0/3001920036 >> v2:172.21.15.40:3300/0 conn(0x7fe2c80b3ae0 0x7fe2c813dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.149+0000 7fe2ce480640 1 -- 172.21.15.40:0/3001920036 >> v1:172.21.15.40:6789/0 conn(0x7fe2c80a8910 legacy=0x7fe2c8140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.149+0000 7fe2ce480640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.149+0000 7fe2ce480640 1 monclient: found mon.noname-b 2023-07-29T13:28:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.149+0000 7fe2ce480640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.149+0000 7fe2ce480640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.149+0000 7fe2ce480640 1 -- 172.21.15.40:0/3001920036 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80a9c30 con 0x7fe2c8141320 2023-07-29T13:28:17.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.154+0000 7fe2ce480640 1 --2- 172.21.15.40:0/3001920036 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2c8141320 0x7fe2c813e1d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009d90 tx=0x7fe2c002be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8ea974cbf3f9240 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 1 -- 172.21.15.40:0/3001920036 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c002cc00 con 0x7fe2c8141320 2023-07-29T13:28:17.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 10 monclient: dump: 2023-07-29T13:28:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.164 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 1 -- 172.21.15.40:0/3001920036 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c002cd60 con 0x7fe2c8141320 2023-07-29T13:28:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2cfee3640 5 monclient: authenticate success, global_id 4244 2023-07-29T13:28:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2cfee3640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 1 -- 172.21.15.40:0/3001920036 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0034640 con 0x7fe2c8141320 2023-07-29T13:28:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2ccc7d640 10 monclient: dump: 2023-07-29T13:28:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.167 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.168 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.168 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.168 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.168 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2cfee3640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:17.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.160+0000 7fe2cfee3640 4 set_mon_vals no callback set 2023-07-29T13:28:17.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.161+0000 7fe2cfee3640 10 monclient: shutdown 2023-07-29T13:28:17.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.161+0000 7fe2cfee3640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.161+0000 7fe2cfee3640 1 -- 172.21.15.40:0/3001920036 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2c8141320 msgr2=0x7fe2c813e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.161+0000 7fe2cfee3640 1 --2- 172.21.15.40:0/3001920036 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2c8141320 0x7fe2c813e1d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009d90 tx=0x7fe2c002be50 comp rx=0 tx=0).stop 2023-07-29T13:28:17.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.162+0000 7fe2cfee3640 1 -- 172.21.15.40:0/3001920036 shutdown_connections 2023-07-29T13:28:17.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.162+0000 7fe2cfee3640 1 --2- 172.21.15.40:0/3001920036 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2c8141320 0x7fe2c813e1d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.162+0000 7fe2cfee3640 1 --2- 172.21.15.40:0/3001920036 >> v2:172.21.15.40:3300/0 conn(0x7fe2c80b3ae0 0x7fe2c813dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.162+0000 7fe2cfee3640 1 -- 172.21.15.40:0/3001920036 >> 172.21.15.40:0/3001920036 conn(0x7fe2c8070fc0 msgr2=0x7fe2c80713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.162+0000 7fe2cfee3640 1 -- 172.21.15.40:0/3001920036 shutdown_connections 2023-07-29T13:28:17.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.162+0000 7fe2cfee3640 1 -- 172.21.15.40:0/3001920036 wait complete. 2023-07-29T13:28:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.163+0000 7fe2cfee3640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.163+0000 7fe2cfee3640 10 monclient: monmap: 2023-07-29T13:28:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.173 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.173 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:17.173 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:17.173 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:17.173 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.163+0000 7fe2cfee3640 1 librados: starting msgr at 2023-07-29T13:28:17.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.163+0000 7fe2cfee3640 1 librados: starting objecter 2023-07-29T13:28:17.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.163+0000 7fe2cfee3640 1 Processor -- start 2023-07-29T13:28:17.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.163+0000 7fe2cfee3640 1 -- start start 2023-07-29T13:28:17.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.163+0000 7fe2cfee3640 1 librados: setting wanted keys 2023-07-29T13:28:17.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.163+0000 7fe2cfee3640 1 librados: calling monclient init 2023-07-29T13:28:17.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.163+0000 7fe2cfee3640 10 monclient: init 2023-07-29T13:28:17.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.164+0000 7fe2cfee3640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.164+0000 7fe2cfee3640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T13:28:17.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.164+0000 7fe2cfee3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2c80a8910 0x7fe2c8134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.164+0000 7fe2cfee3640 10 monclient(hunting): picked mon.noname-c con 0x7fe2c80a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.164+0000 7fe2cfee3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c80b3ae0 0x7fe2c81d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cfee3640 10 monclient(hunting): picked mon.noname-a con 0x7fe2c80b3ae0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:17.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cfee3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8141320 0x7fe2c8132f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cfee3640 10 monclient(hunting): picked mon.noname-b con 0x7fe2c8141320 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:17.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cfee3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cfee3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c81d34c0 con 0x7fe2c80b3ae0 2023-07-29T13:28:17.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cfee3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cfee3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c81442a0 con 0x7fe2c8141320 2023-07-29T13:28:17.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cfee3640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cfee3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c81356b0 con 0x7fe2c80a8910 2023-07-29T13:28:17.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cfee3640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cfee3640 10 monclient(hunting): authenticate will time out at 1419.298202s 2023-07-29T13:28:17.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2cdc7f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2c80a8910 0x7fe2c8134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.165+0000 7fe2ce480640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8141320 0x7fe2c8132f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2ce480640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8141320 0x7fe2c8132f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:42508/0 (socket says 172.21.15.40:42508) 2023-07-29T13:28:17.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2ce480640 1 -- 172.21.15.40:0/1490025523 learned_addr learned my addr 172.21.15.40:0/1490025523 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2ce480640 10 monclient(hunting): get_auth_request con 0x7fe2c8141320 auth_method 0 2023-07-29T13:28:17.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2ce480640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2ce480640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2ce480640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2cdc7f640 10 monclient(hunting): get_auth_request con 0x7fe2c80a8910 auth_method 0 2023-07-29T13:28:17.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2cdc7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2cdc7f640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2cdc7f640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2ce480640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2ce480640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.166+0000 7fe2ce480640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.167+0000 7fe2cd47e640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c80b3ae0 0x7fe2c81d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.167+0000 7fe2cd47e640 10 monclient(hunting): get_auth_request con 0x7fe2c80b3ae0 auth_method 0 2023-07-29T13:28:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.167+0000 7fe2cd47e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.167+0000 7fe2cd47e640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.167+0000 7fe2cd47e640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.167+0000 7fe2cdc7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.167+0000 7fe2cdc7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.167+0000 7fe2cdc7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.168+0000 7fe2cd47e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.168+0000 7fe2cd47e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.168+0000 7fe2cd47e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.169+0000 7fe2ce480640 10 monclient(hunting): handle_auth_done global_id 4309 payload 824 2023-07-29T13:28:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.169+0000 7fe2ce480640 1 -- 172.21.15.40:0/1490025523 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2c80a8910 msgr2=0x7fe2c8134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.169+0000 7fe2ce480640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2c80a8910 0x7fe2c8134fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.170+0000 7fe2ce480640 1 -- 172.21.15.40:0/1490025523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c80b3ae0 msgr2=0x7fe2c81d58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.170+0000 7fe2ce480640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c80b3ae0 0x7fe2c81d58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.170+0000 7fe2ce480640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.170+0000 7fe2ce480640 1 monclient: found mon.noname-b 2023-07-29T13:28:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.170+0000 7fe2ce480640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.170+0000 7fe2ce480640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.170+0000 7fe2ce480640 1 -- 172.21.15.40:0/1490025523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c0009a70 con 0x7fe2c8141320 2023-07-29T13:28:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.172+0000 7fe2ce480640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8141320 0x7fe2c8132f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0037040 tx=0x7fe2c0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.172+0000 7fe2be7fc640 10 client.?.objecter ms_handle_connect 0x7fe2c8141320 2023-07-29T13:28:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.172+0000 7fe2be7fc640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 1 -- 172.21.15.40:0/1490025523 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0004800 con 0x7fe2c8141320 2023-07-29T13:28:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 10 monclient: dump: 2023-07-29T13:28:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.184 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 1 -- 172.21.15.40:0/1490025523 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c0034440 con 0x7fe2c8141320 2023-07-29T13:28:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 1 -- 172.21.15.40:0/1490025523 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00347f0 con 0x7fe2c8141320 2023-07-29T13:28:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2be7fc640 10 monclient: dump: 2023-07-29T13:28:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.187 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2bf7fe640 4 set_mon_vals no callback set 2023-07-29T13:28:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2cfee3640 5 monclient: authenticate success, global_id 4309 2023-07-29T13:28:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2cfee3640 10 monclient: _renew_subs 2023-07-29T13:28:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2cfee3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.176+0000 7fe2cfee3640 1 -- 172.21.15.40:0/1490025523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c8143ea0 con 0x7fe2c8141320 2023-07-29T13:28:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.177+0000 7fe2cfee3640 10 client.4309.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.177+0000 7fe2cfee3640 10 monclient: _renew_subs 2023-07-29T13:28:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.177+0000 7fe2cfee3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.177+0000 7fe2cfee3640 1 -- 172.21.15.40:0/1490025523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c8143720 con 0x7fe2c8141320 2023-07-29T13:28:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.177+0000 7fe2cfee3640 1 librados: init done 2023-07-29T13:28:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.178+0000 7fd59ebc4640 10 monclient: get_monmap_and_config 2023-07-29T13:28:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.178+0000 7fd59ebc4640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.178+0000 7fd59ebc4640 10 monclient: monmap: 2023-07-29T13:28:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:17.179319+0000 2023-07-29T13:28:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:17.179319+0000 2023-07-29T13:28:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:17.191 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:17.191 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:17.191 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.178+0000 7f44a3fff640 10 monclient: get_monmap_and_config 2023-07-29T13:28:17.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.178+0000 7f44a3fff640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.178+0000 7fe2cfee3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.178+0000 7f44a3fff640 10 monclient: monmap: 2023-07-29T13:28:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:17.179880+0000 2023-07-29T13:28:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:17.179880+0000 2023-07-29T13:28:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:17.194 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:17.194 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:17.194 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.178+0000 7fe2cfee3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.178+0000 7fe2cfee3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:17.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.178+0000 7fe2cfee3640 1 -- 172.21.15.40:0/1490025523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c8144460 con 0x7fe2c8141320 2023-07-29T13:28:17.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.179+0000 7fd59ebc4640 1 Processor -- start 2023-07-29T13:28:17.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.179+0000 7f44a3fff640 1 Processor -- start 2023-07-29T13:28:17.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.179+0000 7fd59ebc4640 1 -- start start 2023-07-29T13:28:17.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.179+0000 7fd59ebc4640 10 monclient: init 2023-07-29T13:28:17.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.179+0000 7f44a3fff640 1 -- start start 2023-07-29T13:28:17.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.179+0000 7f44a3fff640 10 monclient: init 2023-07-29T13:28:17.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7f85d1957640 10 monclient: get_monmap_and_config 2023-07-29T13:28:17.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7f85d1957640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7f85d1957640 10 monclient: monmap: 2023-07-29T13:28:17.197 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.197 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.198 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:17.181230+0000 2023-07-29T13:28:17.198 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:17.181230+0000 2023-07-29T13:28:17.198 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.198 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.198 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:17.198 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:17.198 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:17.199 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 10 monclient: _add_conns ranks=[1,0,4,2,3] 2023-07-29T13:28:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd5980b7130 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd59809cc20 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5980b0d40 0x7fd5980b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 10 monclient(hunting): picked mon.noname-c con 0x7fd5980b0d40 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5980b7500 con 0x7fd59809cc20 2023-07-29T13:28:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5980b21f0 con 0x7fd5980b7130 2023-07-29T13:28:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5980b24d0 con 0x7fd5980b0d40 2023-07-29T13:28:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7fd59ebc4640 10 monclient(hunting): authenticate will time out at 1419.313578s 2023-07-29T13:28:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7f44a3fff640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7f44a3fff640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2023-07-29T13:28:17.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7f44a3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44a40b6f90 0x7f44a4144380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7f44a3fff640 10 monclient(hunting): picked mon.noname-a con 0x7f44a40b6f90 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:17.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7f44a3fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f44a40b7360 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:17.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.180+0000 7f44a3fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f44a4140940 0x7f44a4140d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f44a3fff640 10 monclient(hunting): picked mon.noname-c con 0x7f44a4140940 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:17.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 1 Processor -- start 2023-07-29T13:28:17.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f44a3fff640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a40ae7e0 con 0x7f44a40b7360 2023-07-29T13:28:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f44a3fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f44a3fff640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a40b1060 con 0x7f44a40b6f90 2023-07-29T13:28:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f44a3fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f44a3fff640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a40aea30 con 0x7f44a4140940 2023-07-29T13:28:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f44a3fff640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f44a3fff640 10 monclient(hunting): authenticate will time out at 1419.314289s 2023-07-29T13:28:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 1 -- start start 2023-07-29T13:28:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 10 monclient: init 2023-07-29T13:28:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f5341995640 10 monclient: get_monmap_and_config 2023-07-29T13:28:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f5341995640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f5341995640 10 monclient: monmap: 2023-07-29T13:28:17.206 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.206 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.206 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:17.182491+0000 2023-07-29T13:28:17.206 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:17.182491+0000 2023-07-29T13:28:17.206 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.206 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.206 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:17.207 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:17.207 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:17.207 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:17.207 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:17.207 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2023-07-29T13:28:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7fd59d3c1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd59809cc20 0x7fd5980b0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51292/0 (socket says 172.21.15.40:51292) 2023-07-29T13:28:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7fd59d3c1640 1 -- 172.21.15.40:0/2444700666 learned_addr learned my addr 172.21.15.40:0/2444700666 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:28:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f85cc0b0580 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 10 monclient(hunting): picked mon.noname-b con 0x7f85cc13d550 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85cc141120 0x7f85cc141560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 10 monclient(hunting): picked mon.noname-c con 0x7f85cc141120 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85cc0b0950 con 0x7f85cc0b0580 2023-07-29T13:28:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85cc0aa260 con 0x7f85cc141120 2023-07-29T13:28:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.181+0000 7f85d1957640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0aa460 con 0x7f85cc13d550 2023-07-29T13:28:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.182+0000 7f85d1957640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.182+0000 7f85d1957640 10 monclient(hunting): authenticate will time out at 1419.315089s 2023-07-29T13:28:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.182+0000 7f5341995640 1 Processor -- start 2023-07-29T13:28:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.182+0000 7f5341995640 1 -- start start 2023-07-29T13:28:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.182+0000 7f5341995640 10 monclient: init 2023-07-29T13:28:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.182+0000 7f85cbfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 10 monclient: _add_conns ranks=[1,3,0,2,4] 2023-07-29T13:28:17.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f533c13a9f0 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:17.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f44a2ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44a40b6f90 0x7f44a4144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f533c0adcf0 0x7f533c143f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 10 monclient(hunting): picked mon.noname-a con 0x7f533c0adcf0 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f533c0aff70 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f85cbfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0a8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49092/0 (socket says 172.21.15.40:49092) 2023-07-29T13:28:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f85cbfff640 1 -- 172.21.15.40:0/1249702690 learned_addr learned my addr 172.21.15.40:0/1249702690 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f85cbfff640 10 monclient(hunting): get_auth_request con 0x7f85cc13d550 auth_method 0 2023-07-29T13:28:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f85cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f85cbfff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f85cbfff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533c13adc0 con 0x7f533c0aff70 2023-07-29T13:28:17.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533c0ad810 con 0x7f533c13a9f0 2023-07-29T13:28:17.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533c0af870 con 0x7f533c0adcf0 2023-07-29T13:28:17.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f5341995640 10 monclient(hunting): authenticate will time out at 1419.316220s 2023-07-29T13:28:17.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f85cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f85cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.183+0000 7f85cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f44a2ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44a40b6f90 0x7f44a4144380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60614/0 (socket says 172.21.15.40:60614) 2023-07-29T13:28:17.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f44a2ffd640 1 -- 172.21.15.40:0/4012362084 learned_addr learned my addr 172.21.15.40:0/4012362084 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f44a2ffd640 10 monclient(hunting): get_auth_request con 0x7f44a40b6f90 auth_method 0 2023-07-29T13:28:17.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f44a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f44a2ffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f44a2ffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f85cbfff640 10 monclient(hunting): handle_auth_done global_id 4250 payload 274 2023-07-29T13:28:17.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f85cbfff640 1 -- 172.21.15.40:0/1249702690 >> v2:172.21.15.45:3300/0 conn(0x7f85cc141120 msgr2=0x7f85cc141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f85cbfff640 1 --2- 172.21.15.40:0/1249702690 >> v2:172.21.15.45:3300/0 conn(0x7f85cc141120 0x7f85cc141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f85cbfff640 1 -- 172.21.15.40:0/1249702690 >> v1:172.21.15.45:6789/0 conn(0x7f85cc0b0580 legacy=0x7f85cc13ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f85cbfff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f85cbfff640 1 monclient: found mon.noname-b 2023-07-29T13:28:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f85cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f85cbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.185+0000 7f85cbfff640 1 -- 172.21.15.40:0/1249702690 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc0a9250 con 0x7f85cc13d550 2023-07-29T13:28:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7f44a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7f44a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7f44a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7f533a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f533c0adcf0 0x7f533c143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7fa0aa313640 10 monclient: get_monmap_and_config 2023-07-29T13:28:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7fa0aa313640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7fa0aa313640 10 monclient: monmap: 2023-07-29T13:28:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:17.187841+0000 2023-07-29T13:28:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:17.187841+0000 2023-07-29T13:28:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:17.222 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7f533a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f533c0adcf0 0x7f533c143f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60622/0 (socket says 172.21.15.40:60622) 2023-07-29T13:28:17.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7f533a7fc640 1 -- 172.21.15.40:0/679524947 learned_addr learned my addr 172.21.15.40:0/679524947 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7f533a7fc640 10 monclient(hunting): get_auth_request con 0x7f533c0adcf0 auth_method 0 2023-07-29T13:28:17.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7f533a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7f533a7fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.186+0000 7f533a7fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.187+0000 7f35a3fff640 10 monclient: get_monmap_and_config 2023-07-29T13:28:17.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.187+0000 7f35a3fff640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.187+0000 7f35a3fff640 10 monclient: monmap: 2023-07-29T13:28:17.224 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.224 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.224 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:17.188240+0000 2023-07-29T13:28:17.224 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:17.188240+0000 2023-07-29T13:28:17.224 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.225 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.225 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:17.225 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:17.225 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:17.225 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:17.225 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:17.225 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.187+0000 7fa0aa313640 1 Processor -- start 2023-07-29T13:28:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.187+0000 7fa0aa313640 1 -- start start 2023-07-29T13:28:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.187+0000 7fa0aa313640 10 monclient: init 2023-07-29T13:28:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7f35a3fff640 1 Processor -- start 2023-07-29T13:28:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7f35a3fff640 1 -- start start 2023-07-29T13:28:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7f35a3fff640 10 monclient: init 2023-07-29T13:28:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2023-07-29T13:28:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa09c0074f0 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c0c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 10 monclient(hunting): picked mon.noname-b con 0x7fa09c008aa0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa09c0c7ea0 0x7fa09c0d1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 10 monclient(hunting): picked mon.noname-c con 0x7fa09c0c7ea0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09c0078c0 con 0x7fa09c0074f0 2023-07-29T13:28:17.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c009b30 con 0x7fa09c0c7ea0 2023-07-29T13:28:17.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c008ef0 con 0x7fa09c008aa0 2023-07-29T13:28:17.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7fa0aa313640 10 monclient(hunting): authenticate will time out at 1419.321756s 2023-07-29T13:28:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7f85cbfff640 1 --2- 172.21.15.40:0/1249702690 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0a8b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009e50 tx=0x7f85bc02bdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f94c64cea7aee2b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7f35a3fff640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.188+0000 7f35a3fff640 10 monclient: _add_conns ranks=[1,0,3,4,2] 2023-07-29T13:28:17.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f35a3fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f359c006b70 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:17.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f35a3fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f359c0c7200 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:17.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f35a3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f359c0d0b10 0x7f359c0d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f35a3fff640 10 monclient(hunting): picked mon.noname-a con 0x7f359c0d0b10 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:17.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f35a3fff640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c006f40 con 0x7f359c0c7200 2023-07-29T13:28:17.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f35a3fff640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c0be100 con 0x7f359c006b70 2023-07-29T13:28:17.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f35a3fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f35a3fff640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359c02b550 con 0x7f359c0d0b10 2023-07-29T13:28:17.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f35a3fff640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f35a3fff640 10 monclient(hunting): authenticate will time out at 1419.322348s 2023-07-29T13:28:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f44a2ffd640 10 monclient(hunting): handle_auth_done global_id 4326 payload 274 2023-07-29T13:28:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f44a2ffd640 1 -- 172.21.15.40:0/4012362084 >> v2:172.21.15.45:3300/0 conn(0x7f44a4140940 msgr2=0x7f44a4140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f44a2ffd640 1 --2- 172.21.15.40:0/4012362084 >> v2:172.21.15.45:3300/0 conn(0x7f44a4140940 0x7f44a4140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f44a2ffd640 1 -- 172.21.15.40:0/4012362084 >> v1:172.21.15.45:6789/0 conn(0x7f44a40b7360 legacy=0x7f44a40b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f44a2ffd640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f44a2ffd640 1 monclient: found mon.noname-a 2023-07-29T13:28:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f44a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f44a2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f44a2ffd640 1 -- 172.21.15.40:0/4012362084 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a41449f0 con 0x7f44a40b6f90 2023-07-29T13:28:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f533a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f533a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.189+0000 7f533a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.192+0000 7f533a7fc640 10 monclient(hunting): handle_auth_done global_id 4329 payload 274 2023-07-29T13:28:17.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.192+0000 7f533a7fc640 1 -- 172.21.15.40:0/679524947 >> v1:172.21.15.45:6789/0 conn(0x7f533c13a9f0 legacy=0x7f533c0b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.192+0000 7f533a7fc640 1 -- 172.21.15.40:0/679524947 >> v1:172.21.15.40:6789/0 conn(0x7f533c0aff70 legacy=0x7f533c14b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.192+0000 7f533a7fc640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.192+0000 7f533a7fc640 1 monclient: found mon.noname-a 2023-07-29T13:28:17.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.192+0000 7f533a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.192+0000 7f533a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.192+0000 7f533a7fc640 1 -- 172.21.15.40:0/679524947 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c0afb50 con 0x7f533c0adcf0 2023-07-29T13:28:17.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 1 -- 172.21.15.40:0/1249702690 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc02c960 con 0x7f85cc13d550 2023-07-29T13:28:17.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:17.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 10 monclient: dump: 2023-07-29T13:28:17.237 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.237 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.237 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.237 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.237 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.237 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.238 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.238 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.238 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.238 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 1 -- 172.21.15.40:0/1249702690 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc02cac0 con 0x7f85cc13d550 2023-07-29T13:28:17.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85d1957640 5 monclient: authenticate success, global_id 4250 2023-07-29T13:28:17.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85d1957640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:17.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 1 -- 172.21.15.40:0/1249702690 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc02ce50 con 0x7f85cc13d550 2023-07-29T13:28:17.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:17.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7fa0a8b10640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c0c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85cb7fe640 10 monclient: dump: 2023-07-29T13:28:17.240 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.240 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.240 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.240 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.241 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.241 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.241 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.241 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.241 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.241 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85d1957640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:17.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7f85d1957640 4 set_mon_vals no callback set 2023-07-29T13:28:17.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7fa0a8b10640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c0c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49096/0 (socket says 172.21.15.40:49096) 2023-07-29T13:28:17.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7fa0a8b10640 1 -- 172.21.15.40:0/3142142022 learned_addr learned my addr 172.21.15.40:0/3142142022 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7fa0a8b10640 10 monclient(hunting): get_auth_request con 0x7fa09c008aa0 auth_method 0 2023-07-29T13:28:17.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7fa0a8b10640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7fa0a8b10640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.193+0000 7fa0a8b10640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 10 monclient: shutdown 2023-07-29T13:28:17.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 1 -- 172.21.15.40:0/1249702690 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 msgr2=0x7f85cc0a8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 1 --2- 172.21.15.40:0/1249702690 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0a8b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009e50 tx=0x7f85bc02bdc0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 1 -- 172.21.15.40:0/1249702690 shutdown_connections 2023-07-29T13:28:17.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 1 --2- 172.21.15.40:0/1249702690 >> v2:172.21.15.45:3300/0 conn(0x7f85cc141120 0x7f85cc141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 1 --2- 172.21.15.40:0/1249702690 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0a8b50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 1 -- 172.21.15.40:0/1249702690 >> 172.21.15.40:0/1249702690 conn(0x7f85cc069be0 msgr2=0x7f85cc069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 1 -- 172.21.15.40:0/1249702690 shutdown_connections 2023-07-29T13:28:17.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 1 -- 172.21.15.40:0/1249702690 wait complete. 2023-07-29T13:28:17.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7fa0a8b10640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7fa0a8b10640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7fa0a8b10640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.194+0000 7f85d1957640 10 monclient: monmap: 2023-07-29T13:28:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.248 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:17.248 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:17.248 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:17.248 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.195+0000 7f85d1957640 1 librados: starting msgr at 2023-07-29T13:28:17.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.195+0000 7f85d1957640 1 librados: starting objecter 2023-07-29T13:28:17.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.195+0000 7f85d1957640 1 Processor -- start 2023-07-29T13:28:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.195+0000 7f85d1957640 1 -- start start 2023-07-29T13:28:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.195+0000 7f85d1957640 1 librados: setting wanted keys 2023-07-29T13:28:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.195+0000 7f85d1957640 1 librados: calling monclient init 2023-07-29T13:28:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.195+0000 7f85d1957640 10 monclient: init 2023-07-29T13:28:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T13:28:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85cc0b0580 0x7f85cc065b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 10 monclient(hunting): picked mon.noname-a con 0x7f85cc0b0580 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 10 monclient(hunting): picked mon.noname-c con 0x7f85cc13d550 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85cc141120 0x7f85cc0aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 10 monclient(hunting): picked mon.noname-b con 0x7f85cc141120 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0abf50 con 0x7f85cc0b0580 2023-07-29T13:28:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0ac230 con 0x7f85cc141120 2023-07-29T13:28:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0ac9e0 con 0x7f85cc13d550 2023-07-29T13:28:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7f85d1957640 10 monclient(hunting): authenticate will time out at 1419.329342s 2023-07-29T13:28:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7fa0a8b10640 10 monclient(hunting): handle_auth_done global_id 4253 payload 274 2023-07-29T13:28:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7fa0a8b10640 1 -- 172.21.15.40:0/3142142022 >> v2:172.21.15.45:3300/0 conn(0x7fa09c0c7ea0 msgr2=0x7fa09c0d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7fa0a8b10640 1 --2- 172.21.15.40:0/3142142022 >> v2:172.21.15.45:3300/0 conn(0x7fa09c0c7ea0 0x7fa09c0d1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7fa0a8b10640 1 -- 172.21.15.40:0/3142142022 >> v1:172.21.15.40:6789/0 conn(0x7fa09c0074f0 legacy=0x7fa09c0ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7fa0a8b10640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7fa0a8b10640 1 monclient: found mon.noname-b 2023-07-29T13:28:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7fa0a8b10640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7fa0a8b10640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.196+0000 7fa0a8b10640 1 -- 172.21.15.40:0/3142142022 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c035370 con 0x7fa09c008aa0 2023-07-29T13:28:17.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.198+0000 7f533a7fc640 1 --2- 172.21.15.40:0/679524947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f533c0adcf0 0x7f533c143f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f532c009e50 tx=0x7f532c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9708a2cf13d1634d server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.198+0000 7f44a2ffd640 1 --2- 172.21.15.40:0/4012362084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44a40b6f90 0x7f44a4144380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44a4072040 tx=0x7f449402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d4e0c36c8ff6e server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.199+0000 7fe2be7fc640 1 -- 172.21.15.40:0/1490025523 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fe2c0034950 con 0x7fe2c8141320 2023-07-29T13:28:17.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.200+0000 7fe2be7fc640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe2ac06bfb0 0x7fe2ac06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.200+0000 7fa0a8b10640 1 --2- 172.21.15.40:0/3142142022 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c0c7960 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa094009870 tx=0x7fa09402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd0edcfbc18ae03 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.200+0000 7fe2cdc7f640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe2ac06bfb0 0x7fe2ac06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.200+0000 7fe2cdc7f640 10 monclient: get_auth_request con 0x7fe2ac06bfb0 auth_method 0 2023-07-29T13:28:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.201+0000 7fe2cdc7f640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe2ac06bfb0 0x7fe2ac06e460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009990 tx=0x7fe2c4009110 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.201+0000 7fe2be7fc640 10 client.4309.objecter ms_handle_connect 0x7fe2ac06bfb0 2023-07-29T13:28:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.203+0000 7f85cbfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 1 -- 172.21.15.40:0/3142142022 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094031640 con 0x7fa09c008aa0 2023-07-29T13:28:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 10 monclient: dump: 2023-07-29T13:28:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.258 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 1 -- 172.21.15.40:0/3142142022 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0940317a0 con 0x7fa09c008aa0 2023-07-29T13:28:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 1 -- 172.21.15.40:0/3142142022 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094031b30 con 0x7fa09c008aa0 2023-07-29T13:28:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7fa09bfff640 10 monclient: dump: 2023-07-29T13:28:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.260 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.260 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.260 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.260 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.260 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.260 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.260 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.204+0000 7f85cbfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0ad610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49108/0 (socket says 172.21.15.40:49108) 2023-07-29T13:28:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.205+0000 7f85cbfff640 1 -- 172.21.15.40:0/315626890 learned_addr learned my addr 172.21.15.40:0/315626890 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.205+0000 7f85cbfff640 10 monclient(hunting): get_auth_request con 0x7f85cc13d550 auth_method 0 2023-07-29T13:28:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.205+0000 7f85cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.205+0000 7f85cbfff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.205+0000 7f85cbfff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.205+0000 7f85cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.205+0000 7f85cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.205+0000 7f85cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.205+0000 7fa0aa313640 5 monclient: authenticate success, global_id 4253 2023-07-29T13:28:17.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.205+0000 7fa0aa313640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:17.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.205+0000 7fa0aa313640 4 set_mon_vals no callback set 2023-07-29T13:28:17.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.206+0000 7fa0aa313640 10 monclient: shutdown 2023-07-29T13:28:17.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.206+0000 7fa0aa313640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.206+0000 7fa0aa313640 1 -- 172.21.15.40:0/3142142022 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 msgr2=0x7fa09c0c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.206+0000 7fa0aa313640 1 --2- 172.21.15.40:0/3142142022 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c0c7960 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa094009870 tx=0x7fa09402b8f0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.206+0000 7fa0aa313640 1 -- 172.21.15.40:0/3142142022 shutdown_connections 2023-07-29T13:28:17.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.206+0000 7fa0aa313640 1 --2- 172.21.15.40:0/3142142022 >> v2:172.21.15.45:3300/0 conn(0x7fa09c0c7ea0 0x7fa09c0d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.206+0000 7fa0aa313640 1 --2- 172.21.15.40:0/3142142022 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c0c7960 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.206+0000 7fa0aa313640 1 -- 172.21.15.40:0/3142142022 >> 172.21.15.40:0/3142142022 conn(0x7fa09c037db0 msgr2=0x7fa09c038180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.206+0000 7fa0aa313640 1 -- 172.21.15.40:0/3142142022 shutdown_connections 2023-07-29T13:28:17.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.206+0000 7fa0aa313640 1 -- 172.21.15.40:0/3142142022 wait complete. 2023-07-29T13:28:17.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.207+0000 7fa0aa313640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.207+0000 7fa0aa313640 10 monclient: monmap: 2023-07-29T13:28:17.265 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.265 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:17.265 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:17.265 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:17.265 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.265 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.265 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:17.266 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:17.266 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:17.266 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.207+0000 7f35a37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f359c0d0b10 0x7f359c0d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fa0aa313640 1 librados: starting msgr at 2023-07-29T13:28:17.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fa0aa313640 1 librados: starting objecter 2023-07-29T13:28:17.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fa0aa313640 1 Processor -- start 2023-07-29T13:28:17.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fa0aa313640 1 -- start start 2023-07-29T13:28:17.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fa0aa313640 1 librados: setting wanted keys 2023-07-29T13:28:17.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fa0aa313640 1 librados: calling monclient init 2023-07-29T13:28:17.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fa0aa313640 10 monclient: init 2023-07-29T13:28:17.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076450004 0 0) 0x7fd5980b7500 con 0x7fd59809cc20 2023-07-29T13:28:17.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fd59cbc0640 10 monclient(hunting): _negotiate 2023-07-29T13:28:17.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fd59cbc0640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fd59cbc0640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fd59cbc0640 10 monclient(hunting): my global_id is 4323 2023-07-29T13:28:17.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd584003520 con 0x7fd59809cc20 2023-07-29T13:28:17.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.208+0000 7f85d0955640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85cc0b0580 0x7f85cc065b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T13:28:17.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f35a37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f359c0d0b10 0x7f359c0d2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60626/0 (socket says 172.21.15.40:60626) 2023-07-29T13:28:17.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f35a37fe640 1 -- 172.21.15.40:0/3704712645 learned_addr learned my addr 172.21.15.40:0/3704712645 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f35a37fe640 10 monclient(hunting): get_auth_request con 0x7f359c0d0b10 auth_method 0 2023-07-29T13:28:17.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f35a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f35a37fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f35a37fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa09c0074f0 0x7fa09c0ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 10 monclient(hunting): picked mon.noname-b con 0x7fa09c0074f0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c163d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 10 monclient(hunting): picked mon.noname-c con 0x7fa09c008aa0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa09c0c7ea0 0x7fa09c164270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 10 monclient(hunting): picked mon.noname-a con 0x7fa09c0c7ea0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:17.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c02ff40 con 0x7fa09c0c7ea0 2023-07-29T13:28:17.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0c33a0 con 0x7fa09c0074f0 2023-07-29T13:28:17.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c030140 con 0x7fa09c008aa0 2023-07-29T13:28:17.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7fa0aa313640 10 monclient(hunting): authenticate will time out at 1419.342304s 2023-07-29T13:28:17.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f85cbfff640 10 monclient(hunting): handle_auth_done global_id 4256 payload 824 2023-07-29T13:28:17.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f85cbfff640 1 -- 172.21.15.40:0/315626890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85cc141120 msgr2=0x7f85cc0aaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f85cbfff640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85cc141120 0x7f85cc0aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f85cbfff640 1 -- 172.21.15.40:0/315626890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85cc0b0580 msgr2=0x7f85cc065b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f85cbfff640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85cc0b0580 0x7f85cc065b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f85cbfff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f85cbfff640 1 monclient: found mon.noname-c 2023-07-29T13:28:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f85cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f85cbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.209+0000 7f85cbfff640 1 -- 172.21.15.40:0/315626890 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc009b30 con 0x7f85cc13d550 2023-07-29T13:28:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 1 -- 172.21.15.40:0/679524947 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c02cb20 con 0x7f533c0adcf0 2023-07-29T13:28:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 10 monclient: dump: 2023-07-29T13:28:17.276 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.276 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.276 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.276 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.276 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.276 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.276 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.277 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.277 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.277 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 1 -- 172.21.15.40:0/679524947 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c02cc80 con 0x7f533c0adcf0 2023-07-29T13:28:17.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 1 -- 172.21.15.40:0/679524947 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c0343f0 con 0x7f533c0adcf0 2023-07-29T13:28:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5339ffb640 10 monclient: dump: 2023-07-29T13:28:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.279 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5341995640 5 monclient: authenticate success, global_id 4329 2023-07-29T13:28:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5341995640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.212+0000 7f5341995640 4 set_mon_vals no callback set 2023-07-29T13:28:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f35a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f35a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f35a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f5341995640 10 monclient: shutdown 2023-07-29T13:28:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f5341995640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f5341995640 1 -- 172.21.15.40:0/679524947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f533c0adcf0 msgr2=0x7f533c143f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f5341995640 1 --2- 172.21.15.40:0/679524947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f533c0adcf0 0x7f533c143f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f532c009e50 tx=0x7f532c02bf80 comp rx=0 tx=0).stop 2023-07-29T13:28:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f5341995640 1 -- 172.21.15.40:0/679524947 shutdown_connections 2023-07-29T13:28:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f5341995640 1 --2- 172.21.15.40:0/679524947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f533c0adcf0 0x7f533c143f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f5341995640 1 -- 172.21.15.40:0/679524947 >> 172.21.15.40:0/679524947 conn(0x7f533c06a820 msgr2=0x7f533c0b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f5341995640 1 -- 172.21.15.40:0/679524947 shutdown_connections 2023-07-29T13:28:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.213+0000 7f5341995640 1 -- 172.21.15.40:0/679524947 wait complete. 2023-07-29T13:28:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.214+0000 7fa0a8b10640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.214+0000 7f85cbfff640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0ad610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009e50 tx=0x7f85bc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.214+0000 7f85c97fa640 10 client.?.objecter ms_handle_connect 0x7f85cc13d550 2023-07-29T13:28:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.214+0000 7f85c97fa640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.214+0000 7f5341995640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.214+0000 7f5341995640 10 monclient: monmap: 2023-07-29T13:28:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.285 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.285 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:17.285 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:17.285 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:17.285 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7f5341995640 1 librados: starting msgr at 2023-07-29T13:28:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7f5341995640 1 librados: starting objecter 2023-07-29T13:28:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7f5341995640 1 Processor -- start 2023-07-29T13:28:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7fa0a8b10640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c163d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49118/0 (socket says 172.21.15.40:49118) 2023-07-29T13:28:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7fa0a8b10640 1 -- 172.21.15.40:0/2650277706 learned_addr learned my addr 172.21.15.40:0/2650277706 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7fa0a8b10640 10 monclient(hunting): get_auth_request con 0x7fa09c008aa0 auth_method 0 2023-07-29T13:28:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7fa0a8b10640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7fa0a8b10640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7fa0a8b10640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7f5341995640 1 -- start start 2023-07-29T13:28:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7f5341995640 1 librados: setting wanted keys 2023-07-29T13:28:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7f5341995640 1 librados: calling monclient init 2023-07-29T13:28:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.215+0000 7f5341995640 10 monclient: init 2023-07-29T13:28:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T13:28:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f533c0adcf0 0x7f533c0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 10 monclient(hunting): picked mon.noname-a con 0x7f533c0adcf0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f533c0aff70 0x7f533c1d62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 10 monclient(hunting): picked mon.noname-c con 0x7f533c0aff70 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f533c13a9f0 0x7f533c13ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 10 monclient(hunting): picked mon.noname-b con 0x7f533c13a9f0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c13fd50 con 0x7f533c0adcf0 2023-07-29T13:28:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c13ca60 con 0x7f533c13a9f0 2023-07-29T13:28:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c13c7a0 con 0x7f533c0aff70 2023-07-29T13:28:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f5341995640 10 monclient(hunting): authenticate will time out at 1419.349222s 2023-07-29T13:28:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f35a37fe640 10 monclient(hunting): handle_auth_done global_id 4335 payload 274 2023-07-29T13:28:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f35a37fe640 1 -- 172.21.15.40:0/3704712645 >> v1:172.21.15.45:6789/0 conn(0x7f359c006b70 legacy=0x7f359c033bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f35a37fe640 1 -- 172.21.15.40:0/3704712645 >> v1:172.21.15.40:6789/0 conn(0x7f359c0c7200 legacy=0x7f359c0c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f35a37fe640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f35a37fe640 1 monclient: found mon.noname-a 2023-07-29T13:28:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f35a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f35a37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7f35a37fe640 1 -- 172.21.15.40:0/3704712645 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0368b0 con 0x7f359c0d0b10 2023-07-29T13:28:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7fa0a8b10640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7fa0a8b10640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.216+0000 7fa0a8b10640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 1 -- 172.21.15.40:0/4012362084 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44940317e0 con 0x7f44a40b6f90 2023-07-29T13:28:17.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:17.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 10 monclient: dump: 2023-07-29T13:28:17.294 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.294 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.294 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.295 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.295 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.295 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.295 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.295 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.295 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.295 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 1 -- 172.21.15.40:0/4012362084 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4494031940 con 0x7f44a40b6f90 2023-07-29T13:28:17.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 1 -- 172.21.15.40:0/4012362084 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494031cd0 con 0x7f44a40b6f90 2023-07-29T13:28:17.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a1ffb640 10 monclient: dump: 2023-07-29T13:28:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.298 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.298 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.298 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a3fff640 5 monclient: authenticate success, global_id 4326 2023-07-29T13:28:17.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a3fff640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:17.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.218+0000 7f44a3fff640 4 set_mon_vals no callback set 2023-07-29T13:28:17.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.219+0000 7f44a3fff640 10 monclient: shutdown 2023-07-29T13:28:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.219+0000 7f44a3fff640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.219+0000 7f44a3fff640 1 -- 172.21.15.40:0/4012362084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44a40b6f90 msgr2=0x7f44a4144380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.219+0000 7f44a3fff640 1 --2- 172.21.15.40:0/4012362084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44a40b6f90 0x7f44a4144380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44a4072040 tx=0x7f449402b880 comp rx=0 tx=0).stop 2023-07-29T13:28:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.219+0000 7f44a3fff640 1 -- 172.21.15.40:0/4012362084 shutdown_connections 2023-07-29T13:28:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.219+0000 7f44a3fff640 1 --2- 172.21.15.40:0/4012362084 >> v2:172.21.15.45:3300/0 conn(0x7f44a4140940 0x7f44a4140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.219+0000 7f44a3fff640 1 --2- 172.21.15.40:0/4012362084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44a40b6f90 0x7f44a4144380 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.219+0000 7f44a3fff640 1 -- 172.21.15.40:0/4012362084 >> 172.21.15.40:0/4012362084 conn(0x7f44a406a2b0 msgr2=0x7f44a406a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.219+0000 7f44a3fff640 1 -- 172.21.15.40:0/4012362084 shutdown_connections 2023-07-29T13:28:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.219+0000 7f44a3fff640 1 -- 172.21.15.40:0/4012362084 wait complete. 2023-07-29T13:28:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.220+0000 7f44a3fff640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.220+0000 7f44a3fff640 10 monclient: monmap: 2023-07-29T13:28:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:17.301 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.220+0000 7f44a3fff640 1 librados: starting msgr at 2023-07-29T13:28:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.220+0000 7f44a3fff640 1 librados: starting objecter 2023-07-29T13:28:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.220+0000 7f44a3fff640 1 Processor -- start 2023-07-29T13:28:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.220+0000 7f44a3fff640 1 -- start start 2023-07-29T13:28:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.220+0000 7f44a3fff640 1 librados: setting wanted keys 2023-07-29T13:28:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.220+0000 7f44a3fff640 1 librados: calling monclient init 2023-07-29T13:28:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.220+0000 7f44a3fff640 10 monclient: init 2023-07-29T13:28:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.220+0000 7f35a37fe640 1 --2- 172.21.15.40:0/3704712645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0d0b10 0x7f359c0d2ef0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f35a4069930 tx=0x7f35a40a5640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846c02f28cbd44f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T13:28:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44a40b6f90 0x7f44a409b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 10 monclient(hunting): picked mon.noname-a con 0x7f44a40b6f90 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a40b7360 0x7f44a40ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 10 monclient(hunting): picked mon.noname-b con 0x7f44a40b7360 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:17.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f44a4140940 0x7f44a40acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 10 monclient(hunting): picked mon.noname-c con 0x7f44a4140940 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a409c490 con 0x7f44a40b6f90 2023-07-29T13:28:17.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a409bf70 con 0x7f44a40b7360 2023-07-29T13:28:17.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40ad260 con 0x7f44a4140940 2023-07-29T13:28:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.221+0000 7f44a3fff640 10 monclient(hunting): authenticate will time out at 1419.354680s 2023-07-29T13:28:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.222+0000 7fa0a8b10640 10 monclient(hunting): handle_auth_done global_id 4259 payload 824 2023-07-29T13:28:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.222+0000 7fa0a8b10640 1 -- 172.21.15.40:0/2650277706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa09c0074f0 msgr2=0x7fa09c0ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.222+0000 7fa0a8b10640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa09c0074f0 0x7fa09c0ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.222+0000 7fa0a8b10640 1 -- 172.21.15.40:0/2650277706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa09c0c7ea0 msgr2=0x7fa09c164270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.222+0000 7fa0a8b10640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa09c0c7ea0 0x7fa09c164270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.222+0000 7fa0a8b10640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.222+0000 7fa0a8b10640 1 monclient: found mon.noname-c 2023-07-29T13:28:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.222+0000 7fa0a8b10640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.222+0000 7fa0a8b10640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.222+0000 7fa0a8b10640 1 -- 172.21.15.40:0/2650277706 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0940094d0 con 0x7fa09c008aa0 2023-07-29T13:28:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 1 -- 172.21.15.40:0/315626890 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc03e450 con 0x7f85cc13d550 2023-07-29T13:28:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 10 monclient: dump: 2023-07-29T13:28:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.309 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 1 -- 172.21.15.40:0/315626890 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc03e5b0 con 0x7f85cc13d550 2023-07-29T13:28:17.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 1 -- 172.21.15.40:0/315626890 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc03e960 con 0x7f85cc13d550 2023-07-29T13:28:17.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85d1957640 5 monclient: authenticate success, global_id 4256 2023-07-29T13:28:17.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85d1957640 10 monclient: _renew_subs 2023-07-29T13:28:17.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85d1957640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85d1957640 1 -- 172.21.15.40:0/315626890 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85cc0acc30 con 0x7f85cc13d550 2023-07-29T13:28:17.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85caffd640 4 set_mon_vals no callback set 2023-07-29T13:28:17.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:17.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85c97fa640 10 monclient: dump: 2023-07-29T13:28:17.311 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.311 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.311 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.311 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.312 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85d1957640 10 client.4256.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85d1957640 10 monclient: _renew_subs 2023-07-29T13:28:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85d1957640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85d1957640 1 -- 172.21.15.40:0/315626890 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85cc0aafa0 con 0x7f85cc13d550 2023-07-29T13:28:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.226+0000 7f85d1957640 1 librados: init done 2023-07-29T13:28:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.227+0000 7f44a37fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f44a4140940 0x7f44a40acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.227+0000 7f44a37fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f44a4140940 0x7f44a40acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49146/0 (socket says 172.21.15.40:49146) 2023-07-29T13:28:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f44a37fe640 1 -- 172.21.15.40:0/4197724347 learned_addr learned my addr 172.21.15.40:0/4197724347 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f44a37fe640 10 monclient(hunting): get_auth_request con 0x7f44a4140940 auth_method 0 2023-07-29T13:28:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f44a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f44a37fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f44a37fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f85d1957640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f85d1957640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f85d1957640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f85d1957640 1 -- 172.21.15.40:0/315626890 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85cc0abcb0 con 0x7f85cc13d550 2023-07-29T13:28:17.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f44a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f44a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.228+0000 7f44a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.229+0000 7fa0a8b10640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c163d30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa09402b930 tx=0x7fa0940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.229+0000 7fa099ffb640 10 client.?.objecter ms_handle_connect 0x7fa09c008aa0 2023-07-29T13:28:17.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.229+0000 7fa099ffb640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:17.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.230+0000 7f533a7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f533c0aff70 0x7f533c1d62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.234+0000 7f44a37fe640 10 monclient(hunting): handle_auth_done global_id 4262 payload 824 2023-07-29T13:28:17.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.234+0000 7f44a37fe640 1 -- 172.21.15.40:0/4197724347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a40b7360 msgr2=0x7f44a40ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.234+0000 7f44a37fe640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a40b7360 0x7f44a40ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.234+0000 7f44a37fe640 1 -- 172.21.15.40:0/4197724347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44a40b6f90 msgr2=0x7f44a409b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.234+0000 7f44a37fe640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44a40b6f90 0x7f44a409b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.234+0000 7f44a37fe640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.234+0000 7f44a37fe640 1 monclient: found mon.noname-c 2023-07-29T13:28:17.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.234+0000 7f44a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.234+0000 7f44a37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.234+0000 7f44a37fe640 1 -- 172.21.15.40:0/4197724347 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44940094d0 con 0x7f44a4140940 2023-07-29T13:28:17.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.235+0000 7f533a7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f533c0aff70 0x7f533c1d62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49134/0 (socket says 172.21.15.40:49134) 2023-07-29T13:28:17.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.235+0000 7f533a7fc640 1 -- 172.21.15.40:0/538335310 learned_addr learned my addr 172.21.15.40:0/538335310 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.235+0000 7f533a7fc640 10 monclient(hunting): get_auth_request con 0x7f533c0aff70 auth_method 0 2023-07-29T13:28:17.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.235+0000 7f533a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.235+0000 7f533a7fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.235+0000 7f533a7fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642100308 0 0) 0x7fd584003520 con 0x7fd59809cc20 2023-07-29T13:28:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 >> v2:172.21.15.45:3300/0 conn(0x7fd5980b0d40 msgr2=0x7fd5980b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 --2- 172.21.15.40:0/2444700666 >> v2:172.21.15.45:3300/0 conn(0x7fd5980b0d40 0x7fd5980b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 >> v1:172.21.15.45:6789/0 conn(0x7fd5980b7130 legacy=0x7fd598140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 monclient: found mon.noname-a-legacy 2023-07-29T13:28:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.40:6789/0 2023-07-29T13:28:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980ae8a0 con 0x7fd59809cc20 2023-07-29T13:28:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3319386150 0 0) 0x7fd588001670 con 0x7fd59809cc20 2023-07-29T13:28:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-07-29T13:28:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient: dump: 2023-07-29T13:28:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.324 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.324 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.324 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] but i'm connected to v1:172.21.15.40:6789/0, reconnecting 2023-07-29T13:28:17.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 >> v1:172.21.15.40:6789/0 conn(0x7fd59809cc20 legacy=0x7fd5980b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T13:28:17.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 --2- 172.21.15.40:0/2444700666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5840039c0 0x7fd584003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient(hunting): picked mon.a con 0x7fd5840039c0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 --2- 172.21.15.40:0/2444700666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5840047c0 0x7fd584004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient(hunting): picked mon.b con 0x7fd5840047c0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 --2- 172.21.15.40:0/2444700666 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd584005270 0x7fd58400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient(hunting): picked mon.c con 0x7fd584005270 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980b24d0 con 0x7fd5840039c0 2023-07-29T13:28:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58400c1a0 con 0x7fd5840047c0 2023-07-29T13:28:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd584014b70 con 0x7fd584005270 2023-07-29T13:28:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.236+0000 7fd59cbc0640 10 monclient(hunting): _finish_auth 0 2023-07-29T13:28:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.237+0000 7f533a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.237+0000 7f533a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.237+0000 7f533a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.243+0000 7f533affd640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f533c0adcf0 0x7f533c0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.243+0000 7f44a37fe640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f44a4140940 0x7f44a40acd20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f449c00b970 tx=0x7f449c00be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.243+0000 7f448b7fe640 10 client.?.objecter ms_handle_connect 0x7f44a4140940 2023-07-29T13:28:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.243+0000 7f448b7fe640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.244+0000 7f533affd640 10 monclient(hunting): get_auth_request con 0x7f533c0adcf0 auth_method 0 2023-07-29T13:28:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.244+0000 7f533affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.244+0000 7f533affd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.244+0000 7f533affd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 1 -- 172.21.15.40:0/3704712645 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40aa7d0 con 0x7f359c0d0b10 2023-07-29T13:28:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 10 monclient: dump: 2023-07-29T13:28:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.331 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 1 -- 172.21.15.40:0/3704712645 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a40a8050 con 0x7f359c0d0b10 2023-07-29T13:28:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 1 -- 172.21.15.40:0/3704712645 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40aac30 con 0x7f359c0d0b10 2023-07-29T13:28:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a1ffb640 10 monclient: dump: 2023-07-29T13:28:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.333 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a3fff640 5 monclient: authenticate success, global_id 4335 2023-07-29T13:28:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a3fff640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7f35a3fff640 4 set_mon_vals no callback set 2023-07-29T13:28:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 1 -- 172.21.15.40:0/2650277706 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09403ac60 con 0x7fa09c008aa0 2023-07-29T13:28:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 10 monclient: dump: 2023-07-29T13:28:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.336 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 1 -- 172.21.15.40:0/2650277706 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09403adc0 con 0x7fa09c008aa0 2023-07-29T13:28:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 1 -- 172.21.15.40:0/2650277706 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094044630 con 0x7fa09c008aa0 2023-07-29T13:28:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa099ffb640 10 monclient: dump: 2023-07-29T13:28:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.338 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa09b7fe640 4 set_mon_vals no callback set 2023-07-29T13:28:17.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7f35a3fff640 10 monclient: shutdown 2023-07-29T13:28:17.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7f35a3fff640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7f35a3fff640 1 -- 172.21.15.40:0/3704712645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0d0b10 msgr2=0x7f359c0d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7f35a3fff640 1 --2- 172.21.15.40:0/3704712645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0d0b10 0x7f359c0d2ef0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f35a4069930 tx=0x7f35a40a5640 comp rx=0 tx=0).stop 2023-07-29T13:28:17.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa0aa313640 5 monclient: authenticate success, global_id 4259 2023-07-29T13:28:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa0aa313640 10 monclient: _renew_subs 2023-07-29T13:28:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa0aa313640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.245+0000 7fa0aa313640 1 -- 172.21.15.40:0/2650277706 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c02d710 con 0x7fa09c008aa0 2023-07-29T13:28:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7fa0aa313640 10 client.4259.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7fa0aa313640 10 monclient: _renew_subs 2023-07-29T13:28:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7fa0aa313640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7fa0aa313640 1 -- 172.21.15.40:0/2650277706 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c1647b0 con 0x7fa09c008aa0 2023-07-29T13:28:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7fa0aa313640 1 librados: init done 2023-07-29T13:28:17.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7f35a3fff640 1 -- 172.21.15.40:0/3704712645 shutdown_connections 2023-07-29T13:28:17.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7f35a3fff640 1 --2- 172.21.15.40:0/3704712645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0d0b10 0x7f359c0d2ef0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7f35a3fff640 1 -- 172.21.15.40:0/3704712645 >> 172.21.15.40:0/3704712645 conn(0x7f359c03a150 msgr2=0x7f359c03a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7f35a3fff640 1 -- 172.21.15.40:0/3704712645 shutdown_connections 2023-07-29T13:28:17.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.246+0000 7f35a3fff640 1 -- 172.21.15.40:0/3704712645 wait complete. 2023-07-29T13:28:17.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f35a3fff640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f35a3fff640 10 monclient: monmap: 2023-07-29T13:28:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:17.344 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f35a3fff640 1 librados: starting msgr at 2023-07-29T13:28:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f35a3fff640 1 librados: starting objecter 2023-07-29T13:28:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7fa0aa313640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7fa0aa313640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7fa0aa313640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7fa0aa313640 1 -- 172.21.15.40:0/2650277706 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c02d8a0 con 0x7fa09c008aa0 2023-07-29T13:28:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f35a3fff640 1 Processor -- start 2023-07-29T13:28:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f35a3fff640 1 -- start start 2023-07-29T13:28:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f35a3fff640 1 librados: setting wanted keys 2023-07-29T13:28:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f35a3fff640 1 librados: calling monclient init 2023-07-29T13:28:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f35a3fff640 10 monclient: init 2023-07-29T13:28:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f533affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f533affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.247+0000 7f533affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T13:28:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f359c006b70 0x7f359c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 10 monclient(hunting): picked mon.noname-c con 0x7f359c006b70 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0c7200 0x7f359c15c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 10 monclient(hunting): picked mon.noname-a con 0x7f359c0c7200 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c0d0b10 0x7f359c15f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 10 monclient(hunting): picked mon.noname-b con 0x7f359c0d0b10 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0c2860 con 0x7f359c0c7200 2023-07-29T13:28:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c032fd0 con 0x7f359c0d0b10 2023-07-29T13:28:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c15cf20 con 0x7f359c006b70 2023-07-29T13:28:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.248+0000 7f35a3fff640 10 monclient(hunting): authenticate will time out at 1419.381373s 2023-07-29T13:28:17.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.249+0000 7f533a7fc640 10 monclient(hunting): handle_auth_done global_id 4265 payload 824 2023-07-29T13:28:17.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.249+0000 7f533a7fc640 1 -- 172.21.15.40:0/538335310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f533c13a9f0 msgr2=0x7f533c13ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.249+0000 7f533a7fc640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f533c13a9f0 0x7f533c13ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.249+0000 7f533a7fc640 1 -- 172.21.15.40:0/538335310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f533c0adcf0 msgr2=0x7f533c0702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.249+0000 7f533a7fc640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f533c0adcf0 0x7f533c0702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.249+0000 7f533a7fc640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.249+0000 7f533a7fc640 1 monclient: found mon.noname-c 2023-07-29T13:28:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.249+0000 7f533a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.249+0000 7f533a7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.249+0000 7f533a7fc640 1 -- 172.21.15.40:0/538335310 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c009b30 con 0x7f533c0aff70 2023-07-29T13:28:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.254+0000 7f533a7fc640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f533c0aff70 0x7f533c1d62d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f532c009e50 tx=0x7f532c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.254+0000 7f5323fff640 10 client.?.objecter ms_handle_connect 0x7f533c0aff70 2023-07-29T13:28:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.254+0000 7f5323fff640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.254+0000 7fd59dbc2640 1 --2- 172.21.15.40:0/2444700666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5840039c0 0x7fd584003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.255+0000 7fd59dbc2640 10 monclient(hunting): get_auth_request con 0x7fd5840039c0 auth_method 0 2023-07-29T13:28:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.255+0000 7fd59dbc2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.255+0000 7fd59dbc2640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.255+0000 7fd59dbc2640 10 monclient(hunting): _init_auth already have auth, reseting 2023-07-29T13:28:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.255+0000 7fd59dbc2640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.255+0000 7fd59dbc2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.256+0000 7fd59dbc2640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-07-29T13:28:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 10 monclient(hunting): handle_auth_done global_id 4323 payload 293 2023-07-29T13:28:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 1 -- 172.21.15.40:0/2444700666 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd584005270 msgr2=0x7fd58400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 1 --2- 172.21.15.40:0/2444700666 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd584005270 0x7fd58400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 1 -- 172.21.15.40:0/2444700666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5840047c0 msgr2=0x7fd584004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 1 --2- 172.21.15.40:0/2444700666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5840047c0 0x7fd584004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 1 monclient: found mon.a 2023-07-29T13:28:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 1 -- 172.21.15.40:0/2444700666 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd584003520 con 0x7fd5840039c0 2023-07-29T13:28:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59dbc2640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59e3c3640 1 -- 172.21.15.40:0/2444700666 reap_dead start 2023-07-29T13:28:17.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59ebc4640 5 monclient: authenticate success, global_id 4323 2023-07-29T13:28:17.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.257+0000 7fd59ebc4640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:17.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.258+0000 7f35a27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0c7200 0x7f359c15c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.259+0000 7f35a27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0c7200 0x7f359c15c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60672/0 (socket says 172.21.15.40:60672) 2023-07-29T13:28:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.259+0000 7f35a27fc640 1 -- 172.21.15.40:0/925906737 learned_addr learned my addr 172.21.15.40:0/925906737 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.259+0000 7f35a27fc640 10 monclient(hunting): get_auth_request con 0x7f359c0c7200 auth_method 0 2023-07-29T13:28:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.259+0000 7f35a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.259+0000 7f35a27fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.259+0000 7f35a27fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.259+0000 7f35a27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.259+0000 7f35a27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.259+0000 7f35a27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 1 -- 172.21.15.40:0/4197724347 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c00cb40 con 0x7f44a4140940 2023-07-29T13:28:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 10 monclient: dump: 2023-07-29T13:28:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.360 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f44a3fff640 5 monclient: authenticate success, global_id 4262 2023-07-29T13:28:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 1 -- 172.21.15.40:0/4197724347 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c00ccc0 con 0x7f44a4140940 2023-07-29T13:28:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 1 -- 172.21.15.40:0/4197724347 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c014680 con 0x7f44a4140940 2023-07-29T13:28:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f44a3fff640 10 monclient: _renew_subs 2023-07-29T13:28:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f44a3fff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f44a3fff640 1 -- 172.21.15.40:0/4197724347 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a4143aa0 con 0x7f44a4140940 2023-07-29T13:28:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f448b7fe640 10 monclient: dump: 2023-07-29T13:28:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.363 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.363 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.363 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.363 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.363 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f44a17fa640 4 set_mon_vals no callback set 2023-07-29T13:28:17.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f44a3fff640 10 client.4262.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:17.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f44a3fff640 10 monclient: _renew_subs 2023-07-29T13:28:17.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f44a3fff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f44a3fff640 1 -- 172.21.15.40:0/4197724347 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a409b6a0 con 0x7f44a4140940 2023-07-29T13:28:17.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f44a3fff640 1 librados: init done 2023-07-29T13:28:17.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f35a27fc640 10 monclient(hunting): handle_auth_done global_id 4341 payload 824 2023-07-29T13:28:17.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f35a27fc640 1 -- 172.21.15.40:0/925906737 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f359c006b70 msgr2=0x7f359c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f35a27fc640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f359c006b70 0x7f359c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f35a27fc640 1 -- 172.21.15.40:0/925906737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c0d0b10 msgr2=0x7f359c15f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f35a27fc640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c0d0b10 0x7f359c15f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f35a27fc640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f35a27fc640 1 monclient: found mon.noname-a 2023-07-29T13:28:17.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f35a27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f35a27fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.262+0000 7f35a27fc640 1 -- 172.21.15.40:0/925906737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c15d0e0 con 0x7f359c0c7200 2023-07-29T13:28:17.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.263+0000 7f44a3fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.263+0000 7f44a3fff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.263+0000 7f44a3fff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.263+0000 7f44a3fff640 1 -- 172.21.15.40:0/4197724347 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a4069be0 con 0x7f44a4140940 2023-07-29T13:28:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.267+0000 7f35a27fc640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0c7200 0x7f359c15c9b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f359800c410 tx=0x7f359800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.267+0000 7f358bfff640 10 client.?.objecter ms_handle_connect 0x7f359c0c7200 2023-07-29T13:28:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.267+0000 7f358bfff640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.271+0000 7fd59dbc2640 1 --2- 172.21.15.40:0/2444700666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5840039c0 0x7fd584003db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd594001650 tx=0x7fd5840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf82f6df6034dba server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 1 -- 172.21.15.40:0/925906737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980035f0 con 0x7f359c0c7200 2023-07-29T13:28:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 10 monclient: dump: 2023-07-29T13:28:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.370 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 1 -- 172.21.15.40:0/925906737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598003750 con 0x7f359c0c7200 2023-07-29T13:28:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 1 -- 172.21.15.40:0/925906737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598003b00 con 0x7f359c0c7200 2023-07-29T13:28:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f35a3fff640 5 monclient: authenticate success, global_id 4341 2023-07-29T13:28:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f35a3fff640 10 monclient: _renew_subs 2023-07-29T13:28:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f35a3fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f35a3fff640 1 -- 172.21.15.40:0/925906737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c0c2280 con 0x7f359c0c7200 2023-07-29T13:28:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f358bfff640 10 monclient: dump: 2023-07-29T13:28:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.373 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f35a17fa640 4 set_mon_vals no callback set 2023-07-29T13:28:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f35a3fff640 10 client.4341.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f35a3fff640 10 monclient: _renew_subs 2023-07-29T13:28:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f35a3fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f35a3fff640 1 -- 172.21.15.40:0/925906737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c033510 con 0x7f359c0c7200 2023-07-29T13:28:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.273+0000 7f35a3fff640 1 librados: init done 2023-07-29T13:28:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.274+0000 7f35a3fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.274+0000 7f35a3fff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.274+0000 7f35a3fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.274+0000 7f35a3fff640 1 -- 172.21.15.40:0/925906737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359c0056b0 con 0x7f359c0c7200 2023-07-29T13:28:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594005270 con 0x7fd5840039c0 2023-07-29T13:28:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59cbc0640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59cbc0640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59cbc0640 10 monclient: dump: 2023-07-29T13:28:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.378 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59402e070 con 0x7fd5840039c0 2023-07-29T13:28:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59cbc0640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59cbc0640 1 -- 172.21.15.40:0/2444700666 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594003610 con 0x7fd5840039c0 2023-07-29T13:28:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59ebc4640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59ebc4640 4 set_mon_vals no callback set 2023-07-29T13:28:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59cbc0640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59cbc0640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59cbc0640 10 monclient: dump: 2023-07-29T13:28:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.380 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59ebc4640 10 monclient: shutdown 2023-07-29T13:28:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59ebc4640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59ebc4640 1 -- 172.21.15.40:0/2444700666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5840039c0 msgr2=0x7fd584003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.277+0000 7fd59ebc4640 1 --2- 172.21.15.40:0/2444700666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5840039c0 0x7fd584003db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd594001650 tx=0x7fd5840057f0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.278+0000 7fd59ebc4640 1 -- 172.21.15.40:0/2444700666 shutdown_connections 2023-07-29T13:28:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.278+0000 7fd59ebc4640 1 --2- 172.21.15.40:0/2444700666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5840039c0 0x7fd584003db0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.278+0000 7fd59ebc4640 1 -- 172.21.15.40:0/2444700666 >> 172.21.15.40:0/2444700666 conn(0x7fd59806a2b0 msgr2=0x7fd59806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.278+0000 7fd59ebc4640 1 -- 172.21.15.40:0/2444700666 shutdown_connections 2023-07-29T13:28:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.278+0000 7fd59ebc4640 1 -- 172.21.15.40:0/2444700666 wait complete. 2023-07-29T13:28:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.278+0000 7fd59ebc4640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.278+0000 7fd59ebc4640 10 monclient: monmap: 2023-07-29T13:28:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:17.383 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 librados: starting msgr at 2023-07-29T13:28:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 librados: starting objecter 2023-07-29T13:28:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 Processor -- start 2023-07-29T13:28:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 -- start start 2023-07-29T13:28:17.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 librados: setting wanted keys 2023-07-29T13:28:17.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 librados: calling monclient init 2023-07-29T13:28:17.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 10 monclient: init 2023-07-29T13:28:17.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T13:28:17.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5840039c0 0x7fd59813b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 10 monclient(hunting): picked mon.noname-b con 0x7fd5840039c0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd59809c100 0x7fd5981d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 10 monclient(hunting): picked mon.noname-c con 0x7fd59809c100 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5981d1aa0 0x7fd5981d4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 10 monclient(hunting): picked mon.noname-a con 0x7fd5981d1aa0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981d1f10 con 0x7fd5981d1aa0 2023-07-29T13:28:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981d1610 con 0x7fd5840039c0 2023-07-29T13:28:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981d49a0 con 0x7fd59809c100 2023-07-29T13:28:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.279+0000 7fd59ebc4640 10 monclient(hunting): authenticate will time out at 1419.413042s 2023-07-29T13:28:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.305+0000 7f0c2affd640 1 -- 172.21.15.40:0/762864763 <== osd.4 v2:172.21.15.45:6826/85183 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0c1c074df0 con 0x7f0c1c072330 2023-07-29T13:28:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.305+0000 7f0c2affd640 10 client.4241.objecter ms_dispatch 0x7f0c3c000b90 command_reply(tid 1: 0 ) v1 2023-07-29T13:28:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.305+0000 7f0c2affd640 10 client.4241.objecter _finish_command 1 = system:0 2023-07-29T13:28:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.305+0000 7f0c2affd640 20 client.4241.objecter put_session s=0x7f0c1c071c60 osd=4 4 2023-07-29T13:28:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.305+0000 7f0c2affd640 15 client.4241.objecter _session_command_op_remove 4 1 2023-07-29T13:28:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.335+0000 7f0c40dd8640 10 client.4241.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-07-29T13:28:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.335+0000 7f0c40dd8640 15 client.4241.objecter _session_command_op_assign -1 2 2023-07-29T13:28:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.335+0000 7f0c40dd8640 20 client.4241.objecter _get_session s=0x7f0c1c071c60 osd=4 3 2023-07-29T13:28:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.335+0000 7f0c40dd8640 20 client.4241.objecter put_session s=0x7f0c1c071c60 osd=4 3 2023-07-29T13:28:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.335+0000 7f0c40dd8640 20 client.4241.objecter _get_session s=0x7f0c1c071c60 osd=4 3 2023-07-29T13:28:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.335+0000 7f0c40dd8640 15 client.4241.objecter _session_command_op_remove -1 2 2023-07-29T13:28:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.335+0000 7f0c40dd8640 20 client.4241.objecter get_session s=0x7f0c1c071c60 osd=4 3 2023-07-29T13:28:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.335+0000 7f0c40dd8640 15 client.4241.objecter _session_command_op_assign 4 2 2023-07-29T13:28:17.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.335+0000 7f0c40dd8640 20 client.4241.objecter put_session s=0x7f0c1c071c60 osd=4 4 2023-07-29T13:28:17.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.335+0000 7f0c40dd8640 10 client.4241.objecter _send_command 2 2023-07-29T13:28:17.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.335+0000 7f0c40dd8640 1 -- 172.21.15.40:0/762864763 --> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c34036b50 con 0x7f0c1c072330 2023-07-29T13:28:17.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c2affd640 1 -- 172.21.15.40:0/762864763 <== osd.4 v2:172.21.15.45:6826/85183 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0c3c04e570 con 0x7f0c1c072330 2023-07-29T13:28:17.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c2affd640 10 client.4241.objecter ms_dispatch 0x7f0c3c000b90 command_reply(tid 2: 0 ) v1 2023-07-29T13:28:17.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c2affd640 10 client.4241.objecter _finish_command 2 = system:0 2023-07-29T13:28:17.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c2affd640 20 client.4241.objecter put_session s=0x7f0c1c071c60 osd=4 4 2023-07-29T13:28:17.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c2affd640 15 client.4241.objecter _session_command_op_remove 4 2 2023-07-29T13:28:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 10 librados: watch_flush enter 2023-07-29T13:28:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 10 librados: watch_flush exit 2023-07-29T13:28:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 10 client.4241.objecter close_session for osd.4 2023-07-29T13:28:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 1 -- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] conn(0x7f0c1c072330 msgr2=0x7f0c1c074740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] conn(0x7f0c1c072330 0x7f0c1c074740 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 20 client.4241.objecter put_session s=0x7f0c1c071c60 osd=4 1 2023-07-29T13:28:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 20 client.4241.objecter shutdown clearing up homeless session... 2023-07-29T13:28:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 10 client.4241.objecter successfully canceled tick 2023-07-29T13:28:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 1 -- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f0c1c06bf60 msgr2=0x7f0c1c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f0c1c06bf60 0x7f0c1c06e410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c30036000 tx=0x7f0c3003c040 comp rx=0 tx=0).stop 2023-07-29T13:28:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 10 monclient: shutdown 2023-07-29T13:28:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 1 -- 172.21.15.40:0/762864763 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c340072b0 msgr2=0x7f0c3402e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 1 --2- 172.21.15.40:0/762864763 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0c340072b0 0x7f0c3402e880 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c820 tx=0x7f0c2c00cce0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c3ad9d640 1 -- 172.21.15.40:0/762864763 reap_dead start 2023-07-29T13:28:17.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 1 -- 172.21.15.40:0/762864763 shutdown_connections 2023-07-29T13:28:17.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 1 -- 172.21.15.40:0/762864763 >> 172.21.15.40:0/762864763 conn(0x7f0c34039e90 msgr2=0x7f0c340384e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 1 -- 172.21.15.40:0/762864763 shutdown_connections 2023-07-29T13:28:17.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 1 -- 172.21.15.40:0/762864763 wait complete. 2023-07-29T13:28:17.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.342+0000 7f0c40dd8640 1 librados: shutdown 2023-07-29T13:28:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:68719476740 2023-07-29T13:28:17.394 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-29T13:28:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.438+0000 7fe2be7fc640 1 -- 172.21.15.40:0/1490025523 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe2c007ead0 con 0x7fe2c8141320 2023-07-29T13:28:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.438+0000 7fe2be7fc640 10 client.4309.objecter ms_dispatch 0x7fe2c8000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.438+0000 7fe2be7fc640 3 client.4309.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.438+0000 7fe2be7fc640 3 client.4309.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.438+0000 7fe2be7fc640 20 client.4309.objecter dump_active .. 0 homeless 2023-07-29T13:28:17.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.451+0000 7fe2be7fc640 1 -- 172.21.15.40:0/1490025523 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7fe2c0076f00 con 0x7fe2c8141320 2023-07-29T13:28:17.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.451+0000 7fe2be7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.451+0000 7fe2be7fc640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:17.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5323fff640 1 -- 172.21.15.40:0/538335310 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c007e70 con 0x7f533c0aff70 2023-07-29T13:28:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5323fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5323fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5323fff640 10 monclient: dump: 2023-07-29T13:28:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.468 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.468 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.468 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.468 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.468 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.468 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.468 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5323fff640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5323fff640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5323fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5341995640 5 monclient: authenticate success, global_id 4265 2023-07-29T13:28:17.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5341995640 10 monclient: _renew_subs 2023-07-29T13:28:17.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5341995640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5341995640 1 -- 172.21.15.40:0/538335310 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c13d4b0 con 0x7f533c0aff70 2023-07-29T13:28:17.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5341995640 10 client.4265.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:17.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5341995640 10 monclient: _renew_subs 2023-07-29T13:28:17.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5341995640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5341995640 1 -- 172.21.15.40:0/538335310 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c13da10 con 0x7f533c0aff70 2023-07-29T13:28:17.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.465+0000 7f5341995640 1 librados: init done 2023-07-29T13:28:17.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.466+0000 7f5341995640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.466+0000 7f5341995640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.466+0000 7f5341995640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.466+0000 7f5341995640 1 -- 172.21.15.40:0/538335310 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533c13cba0 con 0x7f533c0aff70 2023-07-29T13:28:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.467+0000 7fd59dbc2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5840039c0 0x7fd59813b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.467+0000 7fd59dbc2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5840039c0 0x7fd59813b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:42580/0 (socket says 172.21.15.40:42580) 2023-07-29T13:28:17.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.467+0000 7fd59dbc2640 1 -- 172.21.15.40:0/126649436 learned_addr learned my addr 172.21.15.40:0/126649436 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.467+0000 7fd59dbc2640 10 monclient(hunting): get_auth_request con 0x7fd5840039c0 auth_method 0 2023-07-29T13:28:17.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.467+0000 7fd59dbc2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.467+0000 7fd59dbc2640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.467+0000 7fd59dbc2640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.471+0000 7fd59dbc2640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.471+0000 7fd59dbc2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.471+0000 7fd59dbc2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.475+0000 7fd59dbc2640 10 monclient(hunting): handle_auth_done global_id 4312 payload 824 2023-07-29T13:28:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.475+0000 7fd59dbc2640 1 -- 172.21.15.40:0/126649436 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd59809c100 msgr2=0x7fd5981d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.475+0000 7fd59dbc2640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd59809c100 0x7fd5981d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.475+0000 7fd59dbc2640 1 -- 172.21.15.40:0/126649436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5981d1aa0 msgr2=0x7fd5981d4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.475+0000 7fd59dbc2640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5981d1aa0 0x7fd5981d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.475+0000 7fd59dbc2640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.475+0000 7fd59dbc2640 1 monclient: found mon.noname-b 2023-07-29T13:28:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.475+0000 7fd59dbc2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.475+0000 7fd59dbc2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.475+0000 7fd59dbc2640 1 -- 172.21.15.40:0/126649436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5981d4b60 con 0x7fd5840039c0 2023-07-29T13:28:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.480+0000 7fd59dbc2640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5840039c0 0x7fd59813b7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd594001770 tx=0x7fd594000fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.480+0000 7fd58e7fc640 10 client.?.objecter ms_handle_connect 0x7fd5840039c0 2023-07-29T13:28:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.480+0000 7fd58e7fc640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:17.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 1 -- 172.21.15.40:0/126649436 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594007240 con 0x7fd5840039c0 2023-07-29T13:28:17.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:17.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 10 monclient: dump: 2023-07-29T13:28:17.488 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.490 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 1 -- 172.21.15.40:0/126649436 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5940073a0 con 0x7fd5840039c0 2023-07-29T13:28:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd59ebc4640 5 monclient: authenticate success, global_id 4312 2023-07-29T13:28:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd59ebc4640 10 monclient: _renew_subs 2023-07-29T13:28:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd59ebc4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd59ebc4640 1 -- 172.21.15.40:0/126649436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59813aa00 con 0x7fd5840039c0 2023-07-29T13:28:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 1 -- 172.21.15.40:0/126649436 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940067b0 con 0x7fd5840039c0 2023-07-29T13:28:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58e7fc640 10 monclient: dump: 2023-07-29T13:28:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.492 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.492 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.492 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.492 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.492 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.492 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.492 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.492 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.493 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd59ebc4640 10 client.4312.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:17.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd59ebc4640 10 monclient: _renew_subs 2023-07-29T13:28:17.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd59ebc4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:17.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd59ebc4640 1 -- 172.21.15.40:0/126649436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59813af10 con 0x7fd5840039c0 2023-07-29T13:28:17.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd59ebc4640 1 librados: init done 2023-07-29T13:28:17.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.486+0000 7fd58ffff640 4 set_mon_vals no callback set 2023-07-29T13:28:17.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.487+0000 7fd59ebc4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.487+0000 7fd59ebc4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.487+0000 7fd59ebc4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:17.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.487+0000 7fd59ebc4640 1 -- 172.21.15.40:0/126649436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5980751e0 con 0x7fd5840039c0 2023-07-29T13:28:17.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.507+0000 7fd58e7fc640 1 -- 172.21.15.40:0/126649436 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fd594030030 con 0x7fd5840039c0 2023-07-29T13:28:17.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.507+0000 7fd58e7fc640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd57c06bfb0 0x7fd57c06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.508+0000 7fd59d3c1640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd57c06bfb0 0x7fd57c06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.508+0000 7fd59d3c1640 10 monclient: get_auth_request con 0x7fd57c06bfb0 auth_method 0 2023-07-29T13:28:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.508+0000 7fd59d3c1640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd57c06bfb0 0x7fd57c06e460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd59813da10 tx=0x7fd5880015b0 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.508+0000 7fd58e7fc640 10 client.4312.objecter ms_handle_connect 0x7fd57c06bfb0 2023-07-29T13:28:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.513+0000 7f5323fff640 1 -- 172.21.15.40:0/538335310 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c03e450 con 0x7f533c0aff70 2023-07-29T13:28:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.513+0000 7f5323fff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.513+0000 7f5323fff640 1 -- 172.21.15.40:0/538335310 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c03e800 con 0x7f533c0aff70 2023-07-29T13:28:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.513+0000 7f5323fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.513+0000 7f5323fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.513+0000 7f5323fff640 10 monclient: dump: 2023-07-29T13:28:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.517 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.513+0000 7f53397fa640 4 set_mon_vals no callback set 2023-07-29T13:28:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.519+0000 7f876c7a6640 10 monclient: get_monmap_and_config 2023-07-29T13:28:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.519+0000 7f876c7a6640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.519+0000 7f876c7a6640 10 monclient: monmap: 2023-07-29T13:28:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:17.520764+0000 2023-07-29T13:28:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:17.520764+0000 2023-07-29T13:28:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:17.523 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.520+0000 7f876c7a6640 1 Processor -- start 2023-07-29T13:28:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.520+0000 7f876c7a6640 1 -- start start 2023-07-29T13:28:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.520+0000 7f876c7a6640 10 monclient: init 2023-07-29T13:28:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.520+0000 7f876c7a6640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.520+0000 7f876c7a6640 10 monclient: _add_conns ranks=[2,3,1,4,0] 2023-07-29T13:28:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.520+0000 7f876c7a6640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f87640b3ae0 0x7f876413dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.521+0000 7f876c7a6640 10 monclient(hunting): picked mon.noname-b con 0x7f87640b3ae0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.521+0000 7f876c7a6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87640a8910 0x7f876413e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.521+0000 7f876c7a6640 10 monclient(hunting): picked mon.noname-a con 0x7f87640a8910 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.521+0000 7f876c7a6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8764140c10 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.521+0000 7f876c7a6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87640b3eb0 con 0x7f8764140c10 2023-07-29T13:28:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.521+0000 7f876c7a6640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.521+0000 7f876c7a6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640b0f10 con 0x7f87640a8910 2023-07-29T13:28:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.521+0000 7f876c7a6640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.521+0000 7f876c7a6640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640a9950 con 0x7f87640b3ae0 2023-07-29T13:28:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.521+0000 7f876c7a6640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.521+0000 7f876c7a6640 10 monclient(hunting): authenticate will time out at 1419.654176s 2023-07-29T13:28:17.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.553+0000 7f358bfff640 1 -- 172.21.15.40:0/925906737 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f3598003c60 con 0x7f359c0c7200 2023-07-29T13:28:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.554+0000 7f358bfff640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f359006bf60 0x7f359006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.554+0000 7f35a2ffd640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f359006bf60 0x7f359006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.554+0000 7f35a2ffd640 10 monclient: get_auth_request con 0x7f359006bf60 auth_method 0 2023-07-29T13:28:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.554+0000 7f35a2ffd640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f359006bf60 0x7f359006e410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3594001560 tx=0x7f3594007990 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.554+0000 7f358bfff640 10 client.4341.objecter ms_handle_connect 0x7f359006bf60 2023-07-29T13:28:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.566+0000 7f358bfff640 1 -- 172.21.15.40:0/925906737 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f35980010c0 con 0x7f359c0c7200 2023-07-29T13:28:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.566+0000 7f358bfff640 10 client.4341.objecter ms_dispatch 0x7f35a4000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.566+0000 7f358bfff640 3 client.4341.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:17.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.566+0000 7f358bfff640 3 client.4341.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:17.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.566+0000 7f358bfff640 20 client.4341.objecter dump_active .. 0 homeless 2023-07-29T13:28:17.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.566+0000 7f358bfff640 1 -- 172.21.15.40:0/925906737 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f3598053460 con 0x7f359c0c7200 2023-07-29T13:28:17.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.566+0000 7f358bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.566+0000 7f358bfff640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.567+0000 7f8769d41640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87640a8910 0x7f876413e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.568+0000 7f8769d41640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87640a8910 0x7f876413e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60684/0 (socket says 172.21.15.40:60684) 2023-07-29T13:28:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.568+0000 7f8769d41640 1 -- 172.21.15.40:0/3299033395 learned_addr learned my addr 172.21.15.40:0/3299033395 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.568+0000 7f8769d41640 10 monclient(hunting): get_auth_request con 0x7f87640a8910 auth_method 0 2023-07-29T13:28:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.568+0000 7f8769d41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.568+0000 7f8769d41640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.568+0000 7f8769d41640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.572+0000 7f8769d41640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.572+0000 7f8769d41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.572+0000 7f8769d41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.574+0000 7f8769d41640 10 monclient(hunting): handle_auth_done global_id 4344 payload 274 2023-07-29T13:28:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.574+0000 7f8769d41640 1 -- 172.21.15.40:0/3299033395 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f87640b3ae0 msgr2=0x7f876413dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.574+0000 7f8769d41640 1 --2- 172.21.15.40:0/3299033395 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f87640b3ae0 0x7f876413dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.574+0000 7f8769d41640 1 -- 172.21.15.40:0/3299033395 >> v1:172.21.15.45:6789/0 conn(0x7f8764140c10 legacy=0x7f8764141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.574+0000 7f8769d41640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.574+0000 7f8769d41640 1 monclient: found mon.noname-a 2023-07-29T13:28:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.574+0000 7f8769d41640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.574+0000 7f8769d41640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.574+0000 7f8769d41640 1 -- 172.21.15.40:0/3299033395 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87640a9c30 con 0x7f87640a8910 2023-07-29T13:28:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.581+0000 7f8769d41640 1 --2- 172.21.15.40:0/3299033395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87640a8910 0x7f876413e1d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8754009870 tx=0x7f875402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2433403b30af6a7c server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.640+0000 7f8769540640 1 -- 172.21.15.40:0/3299033395 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754031640 con 0x7f87640a8910 2023-07-29T13:28:17.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:17.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 10 monclient: dump: 2023-07-29T13:28:17.643 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.643 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.643 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.644 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.644 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.644 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.644 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.644 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.644 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.644 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 1 -- 172.21.15.40:0/3299033395 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87540317a0 con 0x7f87640a8910 2023-07-29T13:28:17.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f876c7a6640 5 monclient: authenticate success, global_id 4344 2023-07-29T13:28:17.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f876c7a6640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:17.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f876c7a6640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:17.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f876c7a6640 4 set_mon_vals no callback set 2023-07-29T13:28:17.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 1 -- 172.21.15.40:0/3299033395 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754031b30 con 0x7f87640a8910 2023-07-29T13:28:17.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:17.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f8769540640 10 monclient: dump: 2023-07-29T13:28:17.646 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.647 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.647 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.647 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.647 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.647 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.647 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.647 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.648 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.648 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f876c7a6640 10 monclient: shutdown 2023-07-29T13:28:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f876c7a6640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f876c7a6640 1 -- 172.21.15.40:0/3299033395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87640a8910 msgr2=0x7f876413e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f876c7a6640 1 --2- 172.21.15.40:0/3299033395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87640a8910 0x7f876413e1d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8754009870 tx=0x7f875402b8f0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f876c7a6640 1 -- 172.21.15.40:0/3299033395 shutdown_connections 2023-07-29T13:28:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.641+0000 7f876c7a6640 1 --2- 172.21.15.40:0/3299033395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87640a8910 0x7f876413e1d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.642+0000 7f876c7a6640 1 --2- 172.21.15.40:0/3299033395 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f87640b3ae0 0x7f876413dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.642+0000 7f876c7a6640 1 -- 172.21.15.40:0/3299033395 >> 172.21.15.40:0/3299033395 conn(0x7f8764070fc0 msgr2=0x7f87640713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.642+0000 7f876c7a6640 1 -- 172.21.15.40:0/3299033395 shutdown_connections 2023-07-29T13:28:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.642+0000 7f876c7a6640 1 -- 172.21.15.40:0/3299033395 wait complete. 2023-07-29T13:28:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.643+0000 7f876c7a6640 10 monclient: build_initial_monmap 2023-07-29T13:28:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.643+0000 7f876c7a6640 10 monclient: monmap: 2023-07-29T13:28:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:17.653 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:17.653 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.643+0000 7f876c7a6640 1 librados: starting msgr at 2023-07-29T13:28:17.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.643+0000 7f876c7a6640 1 librados: starting objecter 2023-07-29T13:28:17.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.643+0000 7f876c7a6640 1 Processor -- start 2023-07-29T13:28:17.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.643+0000 7f876c7a6640 1 -- start start 2023-07-29T13:28:17.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.643+0000 7f876c7a6640 1 librados: setting wanted keys 2023-07-29T13:28:17.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.643+0000 7f876c7a6640 1 librados: calling monclient init 2023-07-29T13:28:17.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.643+0000 7f876c7a6640 10 monclient: init 2023-07-29T13:28:17.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:17.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:28:17.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f87640a8910 0x7f8764143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 10 monclient(hunting): picked mon.noname-c con 0x7f87640a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:17.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640b3ae0 0x7f8764138bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 10 monclient(hunting): picked mon.noname-b con 0x7f87640b3ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:17.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8764140c10 0x7f87641d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 10 monclient(hunting): picked mon.noname-a con 0x7f8764140c10 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:17.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8764139550 con 0x7f8764140c10 2023-07-29T13:28:17.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8764139130 con 0x7f87640b3ae0 2023-07-29T13:28:17.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:17.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8764138230 con 0x7f87640a8910 2023-07-29T13:28:17.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:17.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.644+0000 7f876c7a6640 10 monclient(hunting): authenticate will time out at 1419.777800s 2023-07-29T13:28:17.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.645+0000 7f876ad43640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8764140c10 0x7f87641d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.646+0000 7f876ad43640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8764140c10 0x7f87641d5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60696/0 (socket says 172.21.15.40:60696) 2023-07-29T13:28:17.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.646+0000 7f876ad43640 1 -- 172.21.15.40:0/679263566 learned_addr learned my addr 172.21.15.40:0/679263566 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:17.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.646+0000 7f876ad43640 10 monclient(hunting): get_auth_request con 0x7f8764140c10 auth_method 0 2023-07-29T13:28:17.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.646+0000 7f876ad43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:17.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.646+0000 7f876ad43640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:17.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.646+0000 7f876ad43640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:17.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.647+0000 7f876ad43640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:17.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.647+0000 7f876ad43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:17.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.647+0000 7f876ad43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:17.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.650+0000 7f876ad43640 10 monclient(hunting): handle_auth_done global_id 4347 payload 824 2023-07-29T13:28:17.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.650+0000 7f876ad43640 1 -- 172.21.15.40:0/679263566 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f87640a8910 msgr2=0x7f8764143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.650+0000 7f876ad43640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f87640a8910 0x7f8764143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.650+0000 7f876ad43640 1 -- 172.21.15.40:0/679263566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640b3ae0 msgr2=0x7f8764138bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.650+0000 7f876ad43640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640b3ae0 0x7f8764138bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.650+0000 7f876ad43640 10 monclient: _finish_hunting 0 2023-07-29T13:28:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.650+0000 7f876ad43640 1 monclient: found mon.noname-a 2023-07-29T13:28:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.650+0000 7f876ad43640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.650+0000 7f876ad43640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.650+0000 7f876ad43640 1 -- 172.21.15.40:0/679263566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540094d0 con 0x7f8764140c10 2023-07-29T13:28:17.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.653+0000 7f876ad43640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8764140c10 0x7f87641d5a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f875c00b990 tx=0x7f875c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.653+0000 7f875affd640 10 client.?.objecter ms_handle_connect 0x7f8764140c10 2023-07-29T13:28:17.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.653+0000 7f875affd640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.660+0000 7f875affd640 1 -- 172.21.15.40:0/679263566 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00cba0 con 0x7f8764140c10 2023-07-29T13:28:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 10 monclient: dump: 2023-07-29T13:28:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.664 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.664 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.664 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.664 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.664 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.665 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 10 monclient: _finish_auth 0 2023-07-29T13:28:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 10 monclient: _check_auth_tickets 2023-07-29T13:28:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f876c7a6640 5 monclient: authenticate success, global_id 4347 2023-07-29T13:28:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 1 -- 172.21.15.40:0/679263566 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00cd00 con 0x7f8764140c10 2023-07-29T13:28:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f876c7a6640 10 monclient: _renew_subs 2023-07-29T13:28:17.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f876c7a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f876c7a6640 1 -- 172.21.15.40:0/679263566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87641d5fa0 con 0x7f8764140c10 2023-07-29T13:28:17.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:17.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f8768d3f640 4 set_mon_vals no callback set 2023-07-29T13:28:17.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f876c7a6640 10 client.4347.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 1 -- 172.21.15.40:0/679263566 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c014680 con 0x7f8764140c10 2023-07-29T13:28:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f876c7a6640 10 monclient: _renew_subs 2023-07-29T13:28:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f876c7a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f876c7a6640 1 -- 172.21.15.40:0/679263566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87641d6480 con 0x7f8764140c10 2023-07-29T13:28:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f876c7a6640 1 librados: init done 2023-07-29T13:28:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.661+0000 7f875affd640 10 monclient: dump: 2023-07-29T13:28:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:17.670 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.662+0000 7f876c7a6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.662+0000 7f876c7a6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.662+0000 7f876c7a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.662+0000 7f876c7a6640 1 -- 172.21.15.40:0/679263566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87641392f0 con 0x7f8764140c10 2023-07-29T13:28:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.685+0000 7f875affd640 1 -- 172.21.15.40:0/679263566 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f875c01d3b0 con 0x7f8764140c10 2023-07-29T13:28:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.686+0000 7f875affd640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f874806bfb0 0x7f874806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:17.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.686+0000 7f876a542640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f874806bfb0 0x7f874806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:17.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.686+0000 7f876a542640 10 monclient: get_auth_request con 0x7f874806bfb0 auth_method 0 2023-07-29T13:28:17.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.687+0000 7f875affd640 10 client.4347.objecter ms_handle_connect 0x7f874806bfb0 2023-07-29T13:28:17.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.687+0000 7f876a542640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f874806bfb0 0x7f874806e460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8760009710 tx=0x7f8760006c90 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.725+0000 7fe2cfee3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2023-07-29T13:28:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.725+0000 7fe2cfee3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-07-29T13:28:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.725+0000 7fe2cfee3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.725+0000 7fe2cfee3640 1 -- 172.21.15.40:0/1490025523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe2c8144060 con 0x7fe2c8141320 2023-07-29T13:28:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.745+0000 7fd58e7fc640 1 -- 172.21.15.40:0/126649436 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd59407c9b0 con 0x7fd5840039c0 2023-07-29T13:28:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.745+0000 7fd58e7fc640 10 client.4312.objecter ms_dispatch 0x7fd598000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.745+0000 7fd58e7fc640 3 client.4312.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.745+0000 7fd58e7fc640 3 client.4312.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.745+0000 7fd58e7fc640 20 client.4312.objecter dump_active .. 0 homeless 2023-07-29T13:28:17.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.754+0000 7f35a3fff640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2023-07-29T13:28:17.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.754+0000 7f35a3fff640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-07-29T13:28:17.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.754+0000 7f35a3fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:17.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.754+0000 7f35a3fff640 1 -- 172.21.15.40:0/925906737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f359c0c8110 con 0x7f359c0c7200 2023-07-29T13:28:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.756+0000 7fd58e7fc640 1 -- 172.21.15.40:0/126649436 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7fd594074d30 con 0x7fd5840039c0 2023-07-29T13:28:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.756+0000 7fd58e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.756+0000 7fd58e7fc640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:17.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.823+0000 7fe2be7fc640 1 -- 172.21.15.40:0/1490025523 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe2c004c090 con 0x7fe2c8141320 2023-07-29T13:28:17.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.823+0000 7fe2be7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-07-29T13:28:17.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.823+0000 7fe2be7fc640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:17.825 INFO:teuthology.orchestra.run.smithi040.stdout:73014444037 2023-07-29T13:28:17.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.824+0000 7fe2cfee3640 10 librados: watch_flush enter 2023-07-29T13:28:17.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.824+0000 7fe2cfee3640 10 librados: watch_flush exit 2023-07-29T13:28:17.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.824+0000 7fe2cfee3640 20 client.4309.objecter shutdown clearing up homeless session... 2023-07-29T13:28:17.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.824+0000 7fe2cfee3640 10 client.4309.objecter successfully canceled tick 2023-07-29T13:28:17.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 -- 172.21.15.40:0/1490025523 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe2ac06bfb0 msgr2=0x7fe2ac06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe2ac06bfb0 0x7fe2ac06e460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009990 tx=0x7fe2c4009110 comp rx=0 tx=0).stop 2023-07-29T13:28:17.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 10 monclient: shutdown 2023-07-29T13:28:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 -- 172.21.15.40:0/1490025523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8141320 msgr2=0x7fe2c8132f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8141320 0x7fe2c8132f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0037040 tx=0x7fe2c0004a40 comp rx=0 tx=0).stop 2023-07-29T13:28:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 -- 172.21.15.40:0/1490025523 shutdown_connections 2023-07-29T13:28:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe2ac06bfb0 0x7fe2ac06e460 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8141320 0x7fe2c8132f30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c80b3ae0 0x7fe2c81d58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 --2- 172.21.15.40:0/1490025523 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2c80a8910 0x7fe2c8134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 -- 172.21.15.40:0/1490025523 >> 172.21.15.40:0/1490025523 conn(0x7fe2c8070fc0 msgr2=0x7fe2c813df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 -- 172.21.15.40:0/1490025523 shutdown_connections 2023-07-29T13:28:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 -- 172.21.15.40:0/1490025523 wait complete. 2023-07-29T13:28:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.825+0000 7fe2cfee3640 1 librados: shutdown 2023-07-29T13:28:17.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444038 got 73014444037 for osd.1 2023-07-29T13:28:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.936+0000 7fd59ebc4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2023-07-29T13:28:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.936+0000 7fd59ebc4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-07-29T13:28:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.936+0000 7fd59ebc4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.936+0000 7fd59ebc4640 1 -- 172.21.15.40:0/126649436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd5980aedf0 con 0x7fd5840039c0 2023-07-29T13:28:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.938+0000 7f875affd640 1 -- 172.21.15.40:0/679263566 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f875c05bf80 con 0x7f8764140c10 2023-07-29T13:28:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.938+0000 7f875affd640 10 client.4347.objecter ms_dispatch 0x7f8764000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.938+0000 7f875affd640 3 client.4347.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.938+0000 7f875affd640 3 client.4347.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.938+0000 7f875affd640 20 client.4347.objecter dump_active .. 0 homeless 2023-07-29T13:28:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.938+0000 7f875affd640 1 -- 172.21.15.40:0/679263566 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f875c054300 con 0x7f8764140c10 2023-07-29T13:28:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.938+0000 7f875affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.938+0000 7f875affd640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:17.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.939+0000 7fd58e7fc640 1 -- 172.21.15.40:0/126649436 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd594044090 con 0x7fd5840039c0 2023-07-29T13:28:17.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.939+0000 7fd58e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-07-29T13:28:17.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.939+0000 7fd58e7fc640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:17.942 INFO:teuthology.orchestra.run.smithi040.stdout:73014444036 2023-07-29T13:28:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 10 librados: watch_flush enter 2023-07-29T13:28:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 10 librados: watch_flush exit 2023-07-29T13:28:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 20 client.4312.objecter shutdown clearing up homeless session... 2023-07-29T13:28:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 10 client.4312.objecter successfully canceled tick 2023-07-29T13:28:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 1 -- 172.21.15.40:0/126649436 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd57c06bfb0 msgr2=0x7fd57c06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd57c06bfb0 0x7fd57c06e460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd59813da10 tx=0x7fd5880015b0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 10 monclient: shutdown 2023-07-29T13:28:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 1 -- 172.21.15.40:0/126649436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5840039c0 msgr2=0x7fd59813b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5840039c0 0x7fd59813b7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd594001770 tx=0x7fd594000fb0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 1 -- 172.21.15.40:0/126649436 shutdown_connections 2023-07-29T13:28:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd57c06bfb0 0x7fd57c06e460 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5981d1aa0 0x7fd5981d4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd59809c100 0x7fd5981d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 1 --2- 172.21.15.40:0/126649436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5840039c0 0x7fd59813b7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.942+0000 7fd59ebc4640 1 -- 172.21.15.40:0/126649436 >> 172.21.15.40:0/126649436 conn(0x7fd59806a2b0 msgr2=0x7fd59806a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.943+0000 7fd59ebc4640 1 -- 172.21.15.40:0/126649436 shutdown_connections 2023-07-29T13:28:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.943+0000 7fd59ebc4640 1 -- 172.21.15.40:0/126649436 wait complete. 2023-07-29T13:28:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.943+0000 7fd59ebc4640 1 librados: shutdown 2023-07-29T13:28:17.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.953+0000 7f358bfff640 1 -- 172.21.15.40:0/925906737 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3598017070 con 0x7f359c0c7200 2023-07-29T13:28:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.953+0000 7f358bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-07-29T13:28:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.953+0000 7f358bfff640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:17.956 INFO:teuthology.orchestra.run.smithi040.stdout:64424509444 2023-07-29T13:28:17.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 10 librados: watch_flush enter 2023-07-29T13:28:17.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 10 librados: watch_flush exit 2023-07-29T13:28:17.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 20 client.4341.objecter shutdown clearing up homeless session... 2023-07-29T13:28:17.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 10 client.4341.objecter successfully canceled tick 2023-07-29T13:28:17.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 1 -- 172.21.15.40:0/925906737 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f359006bf60 msgr2=0x7f359006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f359006bf60 0x7f359006e410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3594001560 tx=0x7f3594007990 comp rx=0 tx=0).stop 2023-07-29T13:28:17.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 10 monclient: shutdown 2023-07-29T13:28:17.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 1 -- 172.21.15.40:0/925906737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0c7200 msgr2=0x7f359c15c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0c7200 0x7f359c15c9b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f359800c410 tx=0x7f359800c8d0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 1 -- 172.21.15.40:0/925906737 shutdown_connections 2023-07-29T13:28:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f359006bf60 0x7f359006e410 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c0d0b10 0x7f359c15f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0c7200 0x7f359c15c9b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 1 --2- 172.21.15.40:0/925906737 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f359c006b70 0x7f359c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:17.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.956+0000 7f35a3fff640 1 -- 172.21.15.40:0/925906737 >> 172.21.15.40:0/925906737 conn(0x7f359c03a150 msgr2=0x7f359c0c64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:17.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.957+0000 7f35a3fff640 1 -- 172.21.15.40:0/925906737 shutdown_connections 2023-07-29T13:28:17.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.957+0000 7f35a3fff640 1 -- 172.21.15.40:0/925906737 wait complete. 2023-07-29T13:28:17.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:17.957+0000 7f35a3fff640 1 librados: shutdown 2023-07-29T13:28:17.961 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444037 got 73014444036 for osd.2 2023-07-29T13:28:17.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.5 2023-07-29T13:28:18.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.101+0000 7f85c97fa640 1 -- 172.21.15.40:0/315626890 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f85bc03eac0 con 0x7f85cc13d550 2023-07-29T13:28:18.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.102+0000 7f85c97fa640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f85ac06bfb0 0x7f85ac06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:18.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.102+0000 7f85d0955640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f85ac06bfb0 0x7f85ac06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:18.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.102+0000 7f85d0955640 10 monclient: get_auth_request con 0x7f85ac06bfb0 auth_method 0 2023-07-29T13:28:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.103+0000 7f85d0955640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f85ac06bfb0 0x7f85ac06e460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f85c40015d0 tx=0x7f85c4009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.103+0000 7f85c97fa640 10 client.4256.objecter ms_handle_connect 0x7f85ac06bfb0 2023-07-29T13:28:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.114+0000 7f448b7fe640 1 -- 172.21.15.40:0/4197724347 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f449c01d3b0 con 0x7f44a4140940 2023-07-29T13:28:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.115+0000 7f448b7fe640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f449006bfb0 0x7f449006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.115+0000 7f44a2ffd640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f449006bfb0 0x7f449006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.115+0000 7f44a2ffd640 10 monclient: get_auth_request con 0x7f449006bfb0 auth_method 0 2023-07-29T13:28:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.115+0000 7f44a2ffd640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f449006bfb0 0x7f449006e460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4494036000 tx=0x7f4494030000 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.115+0000 7f448b7fe640 10 client.4262.objecter ms_handle_connect 0x7f449006bfb0 2023-07-29T13:28:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.125+0000 7f876c7a6640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2023-07-29T13:28:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.125+0000 7f876c7a6640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-07-29T13:28:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.125+0000 7f876c7a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.125+0000 7f876c7a6640 1 -- 172.21.15.40:0/679263566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f87641d6720 con 0x7f8764140c10 2023-07-29T13:28:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.128+0000 7f85c97fa640 1 -- 172.21.15.40:0/315626890 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f85bc07ee60 con 0x7f85cc13d550 2023-07-29T13:28:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.128+0000 7f85c97fa640 10 client.4256.objecter ms_dispatch 0x7f85cc000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.128+0000 7f85c97fa640 3 client.4256.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.128+0000 7f85c97fa640 3 client.4256.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.128+0000 7f85c97fa640 20 client.4256.objecter dump_active .. 0 homeless 2023-07-29T13:28:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.128+0000 7f85c97fa640 1 -- 172.21.15.40:0/315626890 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f85bc0771e0 con 0x7f85cc13d550 2023-07-29T13:28:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.129+0000 7f85c97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.129+0000 7f85c97fa640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.130+0000 7f875affd640 1 -- 172.21.15.40:0/679263566 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f875c019020 con 0x7f8764140c10 2023-07-29T13:28:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.130+0000 7f875affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-07-29T13:28:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.130+0000 7f875affd640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:18.132 INFO:teuthology.orchestra.run.smithi040.stdout:68719476738 2023-07-29T13:28:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.131+0000 7f876c7a6640 10 librados: watch_flush enter 2023-07-29T13:28:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.131+0000 7f876c7a6640 10 librados: watch_flush exit 2023-07-29T13:28:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.131+0000 7f876c7a6640 20 client.4347.objecter shutdown clearing up homeless session... 2023-07-29T13:28:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.131+0000 7f876c7a6640 10 client.4347.objecter successfully canceled tick 2023-07-29T13:28:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 -- 172.21.15.40:0/679263566 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f874806bfb0 msgr2=0x7f874806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f874806bfb0 0x7f874806e460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8760009710 tx=0x7f8760006c90 comp rx=0 tx=0).stop 2023-07-29T13:28:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 10 monclient: shutdown 2023-07-29T13:28:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 -- 172.21.15.40:0/679263566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8764140c10 msgr2=0x7f87641d5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8764140c10 0x7f87641d5a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f875c00b990 tx=0x7f875c00be50 comp rx=0 tx=0).stop 2023-07-29T13:28:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 -- 172.21.15.40:0/679263566 shutdown_connections 2023-07-29T13:28:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f874806bfb0 0x7f874806e460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640b3ae0 0x7f8764138bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8764140c10 0x7f87641d5a60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 --2- 172.21.15.40:0/679263566 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f87640a8910 0x7f8764143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 -- 172.21.15.40:0/679263566 >> 172.21.15.40:0/679263566 conn(0x7f8764070fc0 msgr2=0x7f876413df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 -- 172.21.15.40:0/679263566 shutdown_connections 2023-07-29T13:28:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 -- 172.21.15.40:0/679263566 wait complete. 2023-07-29T13:28:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.132+0000 7f876c7a6640 1 librados: shutdown 2023-07-29T13:28:18.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476738 for osd.4 2023-07-29T13:28:18.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.150+0000 7f448b7fe640 1 -- 172.21.15.40:0/4197724347 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f449c00ce30 con 0x7f44a4140940 2023-07-29T13:28:18.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.150+0000 7f448b7fe640 10 client.4262.objecter ms_dispatch 0x7f44a4000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:18.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.150+0000 7f448b7fe640 3 client.4262.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:18.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.150+0000 7f448b7fe640 3 client.4262.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.150+0000 7f448b7fe640 20 client.4262.objecter dump_active .. 0 homeless 2023-07-29T13:28:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.150+0000 7f448b7fe640 1 -- 172.21.15.40:0/4197724347 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f449c057d80 con 0x7f44a4140940 2023-07-29T13:28:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.150+0000 7f448b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.150+0000 7f448b7fe640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:18.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.181+0000 7fa099ffb640 1 -- 172.21.15.40:0/2650277706 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fa09402f050 con 0x7fa09c008aa0 2023-07-29T13:28:18.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.181+0000 7fa099ffb640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa08806bf60 0x7fa08806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:18.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.182+0000 7fa0a9311640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa08806bf60 0x7fa08806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:18.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.182+0000 7fa0a9311640 10 monclient: get_auth_request con 0x7fa08806bf60 auth_method 0 2023-07-29T13:28:18.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.182+0000 7fa0a9311640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa08806bf60 0x7fa08806e410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0001560 tx=0x7fa0a0007990 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:18.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.182+0000 7fa099ffb640 10 client.4259.objecter ms_handle_connect 0x7fa08806bf60 2023-07-29T13:28:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.200+0000 7fa099ffb640 1 -- 172.21.15.40:0/2650277706 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa09407e5e0 con 0x7fa09c008aa0 2023-07-29T13:28:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.200+0000 7fa099ffb640 10 client.4259.objecter ms_dispatch 0x7fa0a4000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.200+0000 7fa099ffb640 3 client.4259.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.200+0000 7fa099ffb640 3 client.4259.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.200+0000 7fa099ffb640 20 client.4259.objecter dump_active .. 0 homeless 2023-07-29T13:28:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.200+0000 7fa099ffb640 1 -- 172.21.15.40:0/2650277706 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7fa094076960 con 0x7fa09c008aa0 2023-07-29T13:28:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.200+0000 7fa099ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.200+0000 7fa099ffb640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.216+0000 7f5323fff640 1 -- 172.21.15.40:0/538335310 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f532c03e960 con 0x7f533c0aff70 2023-07-29T13:28:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.217+0000 7f5323fff640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f532806bf60 0x7f532806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.217+0000 7f533affd640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f532806bf60 0x7f532806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.217+0000 7f533affd640 10 monclient: get_auth_request con 0x7f532806bf60 auth_method 0 2023-07-29T13:28:18.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.218+0000 7f5323fff640 10 client.4265.objecter ms_handle_connect 0x7f532806bf60 2023-07-29T13:28:18.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.218+0000 7f533affd640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f532806bf60 0x7f532806e410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5330006fd0 tx=0x7f5330009060 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.336+0000 7f44a3fff640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2023-07-29T13:28:18.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.336+0000 7f44a3fff640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-07-29T13:28:18.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.336+0000 7f44a3fff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:18.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.336+0000 7f44a3fff640 1 -- 172.21.15.40:0/4197724347 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f44a40751e0 con 0x7f44a4140940 2023-07-29T13:28:18.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.384+0000 7fa0aa313640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2023-07-29T13:28:18.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.384+0000 7fa0aa313640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2023-07-29T13:28:18.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.384+0000 7fa0aa313640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:18.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.384+0000 7fa0aa313640 1 -- 172.21.15.40:0/2650277706 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa09c0057f0 con 0x7fa09c008aa0 2023-07-29T13:28:18.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.403+0000 7f85d1957640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2023-07-29T13:28:18.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.403+0000 7f85d1957640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2023-07-29T13:28:18.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.403+0000 7f85d1957640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:18.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.403+0000 7f85d1957640 1 -- 172.21.15.40:0/315626890 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f85cc0ac090 con 0x7f85cc13d550 2023-07-29T13:28:18.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.604+0000 7f5323fff640 1 -- 172.21.15.40:0/538335310 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f532c07ac80 con 0x7f533c0aff70 2023-07-29T13:28:18.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.604+0000 7f5323fff640 10 client.4265.objecter ms_dispatch 0x7f533c000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:18.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.604+0000 7f5323fff640 3 client.4265.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:18.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.604+0000 7f5323fff640 3 client.4265.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:18.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.604+0000 7f5323fff640 20 client.4265.objecter dump_active .. 0 homeless 2023-07-29T13:28:18.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.604+0000 7f5323fff640 1 -- 172.21.15.40:0/538335310 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f532c076fc0 con 0x7f533c0aff70 2023-07-29T13:28:18.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.604+0000 7f5323fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:18.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.604+0000 7f5323fff640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:18.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.673+0000 7f448b7fe640 1 -- 172.21.15.40:0/4197724347 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f449c019070 con 0x7f44a4140940 2023-07-29T13:28:18.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.673+0000 7f448b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-07-29T13:28:18.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.673+0000 7f448b7fe640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:18.676 INFO:teuthology.orchestra.run.smithi040.stdout:73014444038 2023-07-29T13:28:18.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.674+0000 7f85c97fa640 1 -- 172.21.15.40:0/315626890 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f85bc043020 con 0x7f85cc13d550 2023-07-29T13:28:18.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.674+0000 7f85c97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2023-07-29T13:28:18.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.674+0000 7f85c97fa640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:18.677 INFO:teuthology.orchestra.run.smithi040.stdout:60129542150 2023-07-29T13:28:18.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 10 librados: watch_flush enter 2023-07-29T13:28:18.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 10 librados: watch_flush exit 2023-07-29T13:28:18.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 20 client.4262.objecter shutdown clearing up homeless session... 2023-07-29T13:28:18.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 10 client.4262.objecter successfully canceled tick 2023-07-29T13:28:18.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 -- 172.21.15.40:0/4197724347 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f449006bfb0 msgr2=0x7f449006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f449006bfb0 0x7f449006e460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4494036000 tx=0x7f4494030000 comp rx=0 tx=0).stop 2023-07-29T13:28:18.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 10 monclient: shutdown 2023-07-29T13:28:18.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:18.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 -- 172.21.15.40:0/4197724347 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f44a4140940 msgr2=0x7f44a40acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f44a4140940 0x7f44a40acd20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f449c00b970 tx=0x7f449c00be30 comp rx=0 tx=0).stop 2023-07-29T13:28:18.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 -- 172.21.15.40:0/4197724347 shutdown_connections 2023-07-29T13:28:18.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f449006bfb0 0x7f449006e460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a40b7360 0x7f44a40ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f44a4140940 0x7f44a40acd20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 --2- 172.21.15.40:0/4197724347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44a40b6f90 0x7f44a409b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 -- 172.21.15.40:0/4197724347 >> 172.21.15.40:0/4197724347 conn(0x7f44a406a2b0 msgr2=0x7f44a40af130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:18.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 -- 172.21.15.40:0/4197724347 shutdown_connections 2023-07-29T13:28:18.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 -- 172.21.15.40:0/4197724347 wait complete. 2023-07-29T13:28:18.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.675+0000 7f44a3fff640 1 librados: shutdown 2023-07-29T13:28:18.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 10 librados: watch_flush enter 2023-07-29T13:28:18.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 10 librados: watch_flush exit 2023-07-29T13:28:18.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 20 client.4256.objecter shutdown clearing up homeless session... 2023-07-29T13:28:18.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 10 client.4256.objecter successfully canceled tick 2023-07-29T13:28:18.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 -- 172.21.15.40:0/315626890 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f85ac06bfb0 msgr2=0x7f85ac06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f85ac06bfb0 0x7f85ac06e460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f85c40015d0 tx=0x7f85c4009040 comp rx=0 tx=0).stop 2023-07-29T13:28:18.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 10 monclient: shutdown 2023-07-29T13:28:18.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:18.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 -- 172.21.15.40:0/315626890 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 msgr2=0x7f85cc0ad610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0ad610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009e50 tx=0x7f85bc004a40 comp rx=0 tx=0).stop 2023-07-29T13:28:18.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 -- 172.21.15.40:0/315626890 shutdown_connections 2023-07-29T13:28:18.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f85ac06bfb0 0x7f85ac06e460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85cc141120 0x7f85cc0aaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f85cc13d550 0x7f85cc0ad610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 --2- 172.21.15.40:0/315626890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85cc0b0580 0x7f85cc065b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 -- 172.21.15.40:0/315626890 >> 172.21.15.40:0/315626890 conn(0x7f85cc069be0 msgr2=0x7f85cc0ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:18.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 -- 172.21.15.40:0/315626890 shutdown_connections 2023-07-29T13:28:18.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 -- 172.21.15.40:0/315626890 wait complete. 2023-07-29T13:28:18.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.677+0000 7f85d1957640 1 librados: shutdown 2023-07-29T13:28:18.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.678+0000 7fa099ffb640 1 -- 172.21.15.40:0/2650277706 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa094052020 con 0x7fa09c008aa0 2023-07-29T13:28:18.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.678+0000 7fa099ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2023-07-29T13:28:18.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.678+0000 7fa099ffb640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:18.685 INFO:teuthology.orchestra.run.smithi040.stdout:64424509445 2023-07-29T13:28:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 10 librados: watch_flush enter 2023-07-29T13:28:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 10 librados: watch_flush exit 2023-07-29T13:28:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 20 client.4259.objecter shutdown clearing up homeless session... 2023-07-29T13:28:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 10 client.4259.objecter successfully canceled tick 2023-07-29T13:28:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 1 -- 172.21.15.40:0/2650277706 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa08806bf60 msgr2=0x7fa08806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa08806bf60 0x7fa08806e410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0001560 tx=0x7fa0a0007990 comp rx=0 tx=0).stop 2023-07-29T13:28:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 10 monclient: shutdown 2023-07-29T13:28:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 1 -- 172.21.15.40:0/2650277706 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 msgr2=0x7fa09c163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c163d30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa09402b930 tx=0x7fa0940046c0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 1 -- 172.21.15.40:0/2650277706 shutdown_connections 2023-07-29T13:28:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fa08806bf60 0x7fa08806e410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa09c0c7ea0 0x7fa09c164270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa09c008aa0 0x7fa09c163d30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.681+0000 7fa0aa313640 1 --2- 172.21.15.40:0/2650277706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa09c0074f0 0x7fa09c0ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.682+0000 7fa0aa313640 1 -- 172.21.15.40:0/2650277706 >> 172.21.15.40:0/2650277706 conn(0x7fa09c037db0 msgr2=0x7fa09c038180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.682+0000 7fa0aa313640 1 -- 172.21.15.40:0/2650277706 shutdown_connections 2023-07-29T13:28:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.682+0000 7fa0aa313640 1 -- 172.21.15.40:0/2650277706 wait complete. 2023-07-29T13:28:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.682+0000 7fa0aa313640 1 librados: shutdown 2023-07-29T13:28:18.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444038 got 73014444038 for osd.0 2023-07-29T13:28:18.693 DEBUG:teuthology.parallel:result is None 2023-07-29T13:28:18.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2023-07-29T13:28:18.697 DEBUG:teuthology.parallel:result is None 2023-07-29T13:28:18.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.6 2023-07-29T13:28:18.698 DEBUG:teuthology.parallel:result is None 2023-07-29T13:28:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.796+0000 7f5341995640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2023-07-29T13:28:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.796+0000 7f5341995640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2023-07-29T13:28:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.796+0000 7f5341995640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.796+0000 7f5341995640 1 -- 172.21.15.40:0/538335310 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f533c143440 con 0x7f533c0aff70 2023-07-29T13:28:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.800+0000 7f5323fff640 1 -- 172.21.15.40:0/538335310 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f532c07a5e0 con 0x7f533c0aff70 2023-07-29T13:28:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.800+0000 7f5323fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2023-07-29T13:28:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.800+0000 7f5323fff640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:18.802 INFO:teuthology.orchestra.run.smithi040.stdout:73014444037 2023-07-29T13:28:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 10 librados: watch_flush enter 2023-07-29T13:28:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 10 librados: watch_flush exit 2023-07-29T13:28:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 20 client.4265.objecter shutdown clearing up homeless session... 2023-07-29T13:28:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 10 client.4265.objecter successfully canceled tick 2023-07-29T13:28:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 -- 172.21.15.40:0/538335310 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f532806bf60 msgr2=0x7f532806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f532806bf60 0x7f532806e410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5330006fd0 tx=0x7f5330009060 comp rx=0 tx=0).stop 2023-07-29T13:28:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 10 monclient: shutdown 2023-07-29T13:28:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 -- 172.21.15.40:0/538335310 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f533c0aff70 msgr2=0x7f533c1d62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f533c0aff70 0x7f533c1d62d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f532c009e50 tx=0x7f532c004a40 comp rx=0 tx=0).stop 2023-07-29T13:28:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 -- 172.21.15.40:0/538335310 shutdown_connections 2023-07-29T13:28:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f532806bf60 0x7f532806e410 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f533c13a9f0 0x7f533c13ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f533c0aff70 0x7f533c1d62d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 --2- 172.21.15.40:0/538335310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f533c0adcf0 0x7f533c0702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 -- 172.21.15.40:0/538335310 >> 172.21.15.40:0/538335310 conn(0x7f533c06a820 msgr2=0x7f533c139ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 -- 172.21.15.40:0/538335310 shutdown_connections 2023-07-29T13:28:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 -- 172.21.15.40:0/538335310 wait complete. 2023-07-29T13:28:18.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.802+0000 7f5341995640 1 librados: shutdown 2023-07-29T13:28:18.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444037 got 73014444037 for osd.3 2023-07-29T13:28:18.815 DEBUG:teuthology.parallel:result is None 2023-07-29T13:28:18.838 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-29T13:28:18.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.946+0000 7f1cb587b640 10 monclient: get_monmap_and_config 2023-07-29T13:28:18.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.946+0000 7f1cb587b640 10 monclient: build_initial_monmap 2023-07-29T13:28:18.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.946+0000 7f1cb587b640 10 monclient: monmap: 2023-07-29T13:28:18.948 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:18.948 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:18.948 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:18.947239+0000 2023-07-29T13:28:18.948 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:18.947239+0000 2023-07-29T13:28:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:18.950 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:18.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.946+0000 7f1cb587b640 1 Processor -- start 2023-07-29T13:28:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.946+0000 7f1cb587b640 1 -- start start 2023-07-29T13:28:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.946+0000 7f1cb587b640 10 monclient: init 2023-07-29T13:28:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 10 monclient: _add_conns ranks=[4,0,2,3,1] 2023-07-29T13:28:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cb00b3ae0 0x7f1cb013dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 10 monclient(hunting): picked mon.noname-c con 0x7f1cb00b3ae0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1cb00a8910 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb0141320 0x7f1cb013e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 10 monclient(hunting): picked mon.noname-b con 0x7f1cb0141320 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00b3eb0 con 0x7f1cb00a8910 2023-07-29T13:28:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb00b0f10 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00a9950 con 0x7f1cb0141320 2023-07-29T13:28:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.947+0000 7f1cb587b640 10 monclient(hunting): authenticate will time out at 1421.080548s 2023-07-29T13:28:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cb00b3ae0 0x7f1cb013dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caf7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb0141320 0x7f1cb013e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cb00b3ae0 0x7f1cb013dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:42610/0 (socket says 172.21.15.40:42610) 2023-07-29T13:28:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caeffd640 1 -- 172.21.15.40:0/2299586356 learned_addr learned my addr 172.21.15.40:0/2299586356 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caeffd640 10 monclient(hunting): get_auth_request con 0x7f1cb00b3ae0 auth_method 0 2023-07-29T13:28:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caeffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caeffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caf7fe640 10 monclient(hunting): get_auth_request con 0x7f1cb0141320 auth_method 0 2023-07-29T13:28:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:18.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caf7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:18.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caf7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:18.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:18.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.948+0000 7f1caeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:18.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.949+0000 7f1caeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:18.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.949+0000 7f1caf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:18.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.949+0000 7f1caf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:18.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.949+0000 7f1caf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:18.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.950+0000 7f1caeffd640 10 monclient(hunting): handle_auth_done global_id 4315 payload 274 2023-07-29T13:28:18.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.950+0000 7f1caeffd640 1 -- 172.21.15.40:0/2299586356 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb0141320 msgr2=0x7f1cb013e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:18.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.950+0000 7f1caeffd640 1 --2- 172.21.15.40:0/2299586356 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb0141320 0x7f1cb013e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.950+0000 7f1caeffd640 1 -- 172.21.15.40:0/2299586356 >> v1:172.21.15.40:6789/0 conn(0x7f1cb00a8910 legacy=0x7f1cb0140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:18.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.950+0000 7f1caeffd640 10 monclient: _finish_hunting 0 2023-07-29T13:28:18.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.950+0000 7f1caeffd640 1 monclient: found mon.noname-c 2023-07-29T13:28:18.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.950+0000 7f1caeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.950+0000 7f1caeffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:28:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.950+0000 7f1caeffd640 1 -- 172.21.15.40:0/2299586356 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00a9c30 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.950+0000 7f1caf7fe640 1 --2- 172.21.15.40:0/2299586356 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb0141320 0x7f1cb013e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-29T13:28:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.952+0000 7f1caeffd640 1 --2- 172.21.15.40:0/2299586356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb00b3ae0 0x7f1cb013dc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009870 tx=0x7f1ca002b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ba3417ee7b63fe5 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 1 -- 172.21.15.40:0/2299586356 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00317c0 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 10 monclient: dump: 2023-07-29T13:28:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:18.961 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 10 monclient: _finish_auth 0 2023-07-29T13:28:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 10 monclient: _check_auth_tickets 2023-07-29T13:28:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 1 -- 172.21.15.40:0/2299586356 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca0031920 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cb587b640 5 monclient: authenticate success, global_id 4315 2023-07-29T13:28:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cb587b640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cb587b640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cb587b640 4 set_mon_vals no callback set 2023-07-29T13:28:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 1 -- 172.21.15.40:0/2299586356 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0031cb0 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.956+0000 7f1cae7fc640 10 monclient: dump: 2023-07-29T13:28:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:18.964 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:18.964 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:18.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-29T13:28:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.957+0000 7f1cb587b640 10 monclient: shutdown 2023-07-29T13:28:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.957+0000 7f1cb587b640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:18.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.957+0000 7f1cb587b640 1 -- 172.21.15.40:0/2299586356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb00b3ae0 msgr2=0x7f1cb013dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.957+0000 7f1cb587b640 1 --2- 172.21.15.40:0/2299586356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb00b3ae0 0x7f1cb013dc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009870 tx=0x7f1ca002b860 comp rx=0 tx=0).stop 2023-07-29T13:28:18.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.957+0000 7f1cb587b640 1 -- 172.21.15.40:0/2299586356 shutdown_connections 2023-07-29T13:28:18.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.957+0000 7f1cb587b640 1 --2- 172.21.15.40:0/2299586356 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb0141320 0x7f1cb013e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.957+0000 7f1cb587b640 1 --2- 172.21.15.40:0/2299586356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb00b3ae0 0x7f1cb013dc90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.957+0000 7f1cb587b640 1 -- 172.21.15.40:0/2299586356 >> 172.21.15.40:0/2299586356 conn(0x7f1cb0070fc0 msgr2=0x7f1cb00713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:18.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.957+0000 7f1cb587b640 1 -- 172.21.15.40:0/2299586356 shutdown_connections 2023-07-29T13:28:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.957+0000 7f1cb587b640 1 -- 172.21.15.40:0/2299586356 wait complete. 2023-07-29T13:28:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 10 monclient: build_initial_monmap 2023-07-29T13:28:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 10 monclient: monmap: 2023-07-29T13:28:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:18.970 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:18.970 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:18.970 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:18.970 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:18.970 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:18.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 1 librados: starting msgr at 2023-07-29T13:28:18.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 1 librados: starting objecter 2023-07-29T13:28:18.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 1 Processor -- start 2023-07-29T13:28:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 1 -- start start 2023-07-29T13:28:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 1 librados: setting wanted keys 2023-07-29T13:28:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 1 librados: calling monclient init 2023-07-29T13:28:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 10 monclient: init 2023-07-29T13:28:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:28:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb00a8910 0x7f1cb01d60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 10 monclient(hunting): picked mon.noname-c con 0x7f1cb00a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb00b3ae0 0x7f1cb00af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 10 monclient(hunting): picked mon.noname-b con 0x7f1cb00b3ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.958+0000 7f1cb587b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb0141320 0x7f1cb00af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1cb587b640 10 monclient(hunting): picked mon.noname-a con 0x7f1cb0141320 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1cb587b640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1cb587b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00aeef0 con 0x7f1cb0141320 2023-07-29T13:28:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1cb587b640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1cb587b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00adc40 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1cb587b640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1cb587b640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00abdb0 con 0x7f1cb00a8910 2023-07-29T13:28:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1cb587b640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1cb587b640 10 monclient(hunting): authenticate will time out at 1421.092110s 2023-07-29T13:28:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caeffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb00a8910 0x7f1cb01d60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb0141320 0x7f1cb00af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1ca7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb00b3ae0 0x7f1cb00af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caeffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb00a8910 0x7f1cb01d60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49200/0 (socket says 172.21.15.40:49200) 2023-07-29T13:28:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caeffd640 1 -- 172.21.15.40:0/277076583 learned_addr learned my addr 172.21.15.40:0/277076583 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caeffd640 10 monclient(hunting): get_auth_request con 0x7f1cb00a8910 auth_method 0 2023-07-29T13:28:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caeffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caeffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caf7fe640 10 monclient(hunting): get_auth_request con 0x7f1cb0141320 auth_method 0 2023-07-29T13:28:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:18.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caf7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:18.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1caf7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:18.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.959+0000 7f1ca7fff640 10 monclient(hunting): get_auth_request con 0x7f1cb00b3ae0 auth_method 0 2023-07-29T13:28:18.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1ca7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:18.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1ca7fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:18.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1ca7fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:18.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1caeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:18.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1caeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:18.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1caeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:18.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1caf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:18.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1caf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:18.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1caf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:18.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1ca7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:18.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1ca7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:18.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.960+0000 7f1ca7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.963+0000 7f1ca7fff640 10 monclient(hunting): handle_auth_done global_id 4318 payload 824 2023-07-29T13:28:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.963+0000 7f1ca7fff640 1 -- 172.21.15.40:0/277076583 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb00a8910 msgr2=0x7f1cb01d60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.963+0000 7f1ca7fff640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb00a8910 0x7f1cb01d60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.963+0000 7f1ca7fff640 1 -- 172.21.15.40:0/277076583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb0141320 msgr2=0x7f1cb00af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.963+0000 7f1ca7fff640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb0141320 0x7f1cb00af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.963+0000 7f1ca7fff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.963+0000 7f1ca7fff640 1 monclient: found mon.noname-b 2023-07-29T13:28:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.963+0000 7f1ca7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:18.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.963+0000 7f1ca7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:18.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.963+0000 7f1ca7fff640 1 -- 172.21.15.40:0/277076583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8009a70 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.965+0000 7f1ca7fff640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb00b3ae0 0x7f1cb00af310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c410 tx=0x7f1c9800c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:18.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.965+0000 7f1cacff9640 10 client.?.objecter ms_handle_connect 0x7f1cb00b3ae0 2023-07-29T13:28:18.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.965+0000 7f1cacff9640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:18.978 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-29T13:28:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 1 -- 172.21.15.40:0/277076583 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c980035f0 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 10 monclient: dump: 2023-07-29T13:28:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:18.982 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:18.982 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:18.982 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:18.982 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:18.982 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:18.982 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:18.982 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:18.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 10 monclient: _finish_auth 0 2023-07-29T13:28:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 10 monclient: _check_auth_tickets 2023-07-29T13:28:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 1 -- 172.21.15.40:0/277076583 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c98003770 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cb587b640 5 monclient: authenticate success, global_id 4318 2023-07-29T13:28:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cb587b640 10 monclient: _renew_subs 2023-07-29T13:28:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cb587b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cb587b640 1 -- 172.21.15.40:0/277076583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca00094d0 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 1 -- 172.21.15.40:0/277076583 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98003b60 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cacff9640 10 monclient: dump: 2023-07-29T13:28:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:18.985 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:18.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cad7fa640 4 set_mon_vals no callback set 2023-07-29T13:28:18.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cb587b640 10 client.4318.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:18.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cb587b640 10 monclient: _renew_subs 2023-07-29T13:28:18.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cb587b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:18.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cb587b640 1 -- 172.21.15.40:0/277076583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb01d6d10 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.970+0000 7f1cb587b640 1 librados: init done 2023-07-29T13:28:18.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.971+0000 7f1cb587b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:18.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.971+0000 7f1cb587b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:18.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.971+0000 7f1cb587b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:18.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.971+0000 7f1cb587b640 1 -- 172.21.15.40:0/277076583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb00af030 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.996+0000 7f1cacff9640 1 -- 172.21.15.40:0/277076583 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f1c98003cc0 con 0x7f1cb00b3ae0 2023-07-29T13:28:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.996+0000 7f1cacff9640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1c9006bfe0 0x7f1c9006e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:18.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.996+0000 7f1caeffd640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1c9006bfe0 0x7f1c9006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:18.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.996+0000 7f1caeffd640 10 monclient: get_auth_request con 0x7f1c9006bfe0 auth_method 0 2023-07-29T13:28:18.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.997+0000 7f1cacff9640 10 client.4318.objecter ms_handle_connect 0x7f1c9006bfe0 2023-07-29T13:28:18.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:18.997+0000 7f1caeffd640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1c9006bfe0 0x7f1c9006e490 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00090b0 tx=0x7f1ca003d040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:19.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.073+0000 7f4f4c675640 10 monclient: get_monmap_and_config 2023-07-29T13:28:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.073+0000 7f4f4c675640 10 monclient: build_initial_monmap 2023-07-29T13:28:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.073+0000 7f4f4c675640 10 monclient: monmap: 2023-07-29T13:28:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:19.074402+0000 2023-07-29T13:28:19.076 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:19.074402+0000 2023-07-29T13:28:19.076 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:19.076 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.076 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:19.076 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:19.076 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:19.076 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:19.076 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:19.077 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.073+0000 7f4f4c675640 1 Processor -- start 2023-07-29T13:28:19.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.073+0000 7f4f4c675640 1 -- start start 2023-07-29T13:28:19.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.073+0000 7f4f4c675640 10 monclient: init 2023-07-29T13:28:19.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:19.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2023-07-29T13:28:19.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4f440b3ae0 0x7f4f4413dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 10 monclient(hunting): picked mon.noname-c con 0x7f4f440b3ae0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:19.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4f440a8910 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:19.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 0x7f4f4413e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 10 monclient(hunting): picked mon.noname-b con 0x7f4f44141320 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:19.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f440b3eb0 con 0x7f4f440a8910 2023-07-29T13:28:19.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f440b0f10 con 0x7f4f440b3ae0 2023-07-29T13:28:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f440a9950 con 0x7f4f44141320 2023-07-29T13:28:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.074+0000 7f4f4c675640 10 monclient(hunting): authenticate will time out at 1421.207822s 2023-07-29T13:28:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.075+0000 7f4f49c10640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4f440a8910 0x7f4f44140c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51320/0 (socket says 172.21.15.40:51320) 2023-07-29T13:28:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.075+0000 7f4f49c10640 1 -- 172.21.15.40:0/289992439 learned_addr learned my addr 172.21.15.40:0/289992439 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:28:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.075+0000 7f4f4ac12640 1 --2- 172.21.15.40:0/289992439 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 0x7f4f4413e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.075+0000 7f4f4ac12640 10 monclient(hunting): get_auth_request con 0x7f4f44141320 auth_method 0 2023-07-29T13:28:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.075+0000 7f4f4ac12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.075+0000 7f4f4ac12640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.075+0000 7f4f4ac12640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.076+0000 7f4f4ac12640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.076+0000 7f4f4ac12640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.076+0000 7f4f4ac12640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4940f640 1 -- 172.21.15.40:0/289992439 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329141805 0 0) 0x7f4f440b3eb0 con 0x7f4f440a8910 2023-07-29T13:28:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4940f640 10 monclient(hunting): _negotiate 2023-07-29T13:28:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4940f640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4940f640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4940f640 10 monclient(hunting): my global_id is 4353 2023-07-29T13:28:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4940f640 1 -- 172.21.15.40:0/289992439 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f30003520 con 0x7f4f440a8910 2023-07-29T13:28:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4ac12640 10 monclient(hunting): handle_auth_done global_id 4274 payload 274 2023-07-29T13:28:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4ac12640 1 -- 172.21.15.40:0/289992439 >> v2:172.21.15.45:3300/0 conn(0x7f4f440b3ae0 msgr2=0x7f4f4413dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4ac12640 1 --2- 172.21.15.40:0/289992439 >> v2:172.21.15.45:3300/0 conn(0x7f4f440b3ae0 0x7f4f4413dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4ac12640 1 -- 172.21.15.40:0/289992439 >> v1:172.21.15.40:6789/0 conn(0x7f4f440a8910 legacy=0x7f4f44140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4ac12640 10 monclient: _finish_hunting 0 2023-07-29T13:28:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4ac12640 1 monclient: found mon.noname-b 2023-07-29T13:28:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4ac12640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4ac12640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.078+0000 7f4f4ac12640 1 -- 172.21.15.40:0/289992439 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f440a9c30 con 0x7f4f44141320 2023-07-29T13:28:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.080+0000 7f4f4ac12640 1 --2- 172.21.15.40:0/289992439 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 0x7f4f4413e1d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009d90 tx=0x7f4f3c02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10643bea87a2a3c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 1 -- 172.21.15.40:0/289992439 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c02cbc0 con 0x7f4f44141320 2023-07-29T13:28:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 10 monclient: dump: 2023-07-29T13:28:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.088 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 10 monclient: _finish_auth 0 2023-07-29T13:28:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 10 monclient: _check_auth_tickets 2023-07-29T13:28:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 1 -- 172.21.15.40:0/289992439 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c02cd20 con 0x7f4f44141320 2023-07-29T13:28:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4c675640 5 monclient: authenticate success, global_id 4274 2023-07-29T13:28:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4c675640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4c675640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4c675640 4 set_mon_vals no callback set 2023-07-29T13:28:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 1 -- 172.21.15.40:0/289992439 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c034660 con 0x7f4f44141320 2023-07-29T13:28:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.084+0000 7f4f4940f640 10 monclient: dump: 2023-07-29T13:28:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.091 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.085+0000 7f4f4c675640 10 monclient: shutdown 2023-07-29T13:28:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.085+0000 7f4f4c675640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.085+0000 7f4f4c675640 1 -- 172.21.15.40:0/289992439 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 msgr2=0x7f4f4413e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.085+0000 7f4f4c675640 1 --2- 172.21.15.40:0/289992439 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 0x7f4f4413e1d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009d90 tx=0x7f4f3c02be10 comp rx=0 tx=0).stop 2023-07-29T13:28:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.085+0000 7f4f4c675640 1 -- 172.21.15.40:0/289992439 shutdown_connections 2023-07-29T13:28:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.085+0000 7f4f4c675640 1 --2- 172.21.15.40:0/289992439 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 0x7f4f4413e1d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.085+0000 7f4f4c675640 1 --2- 172.21.15.40:0/289992439 >> v2:172.21.15.45:3300/0 conn(0x7f4f440b3ae0 0x7f4f4413dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.085+0000 7f4f4c675640 1 -- 172.21.15.40:0/289992439 >> 172.21.15.40:0/289992439 conn(0x7f4f44070fc0 msgr2=0x7f4f440713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.085+0000 7f4f4c675640 1 -- 172.21.15.40:0/289992439 shutdown_connections 2023-07-29T13:28:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.085+0000 7f4f4c675640 1 -- 172.21.15.40:0/289992439 wait complete. 2023-07-29T13:28:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.086+0000 7f4f4c675640 10 monclient: build_initial_monmap 2023-07-29T13:28:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.086+0000 7f4f4c675640 10 monclient: monmap: 2023-07-29T13:28:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:19.095 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.086+0000 7f4f4c675640 1 librados: starting msgr at 2023-07-29T13:28:19.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.086+0000 7f4f4c675640 1 librados: starting objecter 2023-07-29T13:28:19.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.086+0000 7f4f4c675640 1 Processor -- start 2023-07-29T13:28:19.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.086+0000 7f4f4c675640 1 -- start start 2023-07-29T13:28:19.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.086+0000 7f4f4c675640 1 librados: setting wanted keys 2023-07-29T13:28:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.086+0000 7f4f4c675640 1 librados: calling monclient init 2023-07-29T13:28:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.086+0000 7f4f4c675640 10 monclient: init 2023-07-29T13:28:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 10 monclient: _add_conns ranks=[1,0,2] 2023-07-29T13:28:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f440a8910 0x7f4f440b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 10 monclient(hunting): picked mon.noname-b con 0x7f4f440a8910 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f440b3ae0 0x7f4f441d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 10 monclient(hunting): picked mon.noname-a con 0x7f4f440b3ae0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 0x7f4f441d8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 10 monclient(hunting): picked mon.noname-c con 0x7f4f44141320 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f441d6280 con 0x7f4f440b3ae0 2023-07-29T13:28:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f441d5f60 con 0x7f4f440a8910 2023-07-29T13:28:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f441d67a0 con 0x7f4f44141320 2023-07-29T13:28:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.087+0000 7f4f4c675640 10 monclient(hunting): authenticate will time out at 1421.220382s 2023-07-29T13:28:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.088+0000 7f4f49c10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f440b3ae0 0x7f4f441d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.088+0000 7f4f49c10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f440b3ae0 0x7f4f441d5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60714/0 (socket says 172.21.15.40:60714) 2023-07-29T13:28:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.088+0000 7f4f49c10640 1 -- 172.21.15.40:0/1131062270 learned_addr learned my addr 172.21.15.40:0/1131062270 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.088+0000 7f4f49c10640 10 monclient(hunting): get_auth_request con 0x7f4f440b3ae0 auth_method 0 2023-07-29T13:28:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.088+0000 7f4f49c10640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:19.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.088+0000 7f4f49c10640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:19.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.088+0000 7f4f49c10640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:19.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.089+0000 7f4f4ac12640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 0x7f4f441d8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:19.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.090+0000 7f4f4ac12640 10 monclient(hunting): get_auth_request con 0x7f4f44141320 auth_method 0 2023-07-29T13:28:19.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.090+0000 7f4f4ac12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:19.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.090+0000 7f4f4ac12640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:19.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.090+0000 7f4f4ac12640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:19.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.090+0000 7f4f4ac12640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.090+0000 7f4f4ac12640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.090+0000 7f4f4ac12640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:19.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.092+0000 7f4f49c10640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:19.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.092+0000 7f4f49c10640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:19.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.092+0000 7f4f49c10640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:19.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.093+0000 7f4f4ac12640 10 monclient(hunting): handle_auth_done global_id 4277 payload 824 2023-07-29T13:28:19.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.093+0000 7f4f4ac12640 1 -- 172.21.15.40:0/1131062270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f440a8910 msgr2=0x7f4f440b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.093+0000 7f4f4ac12640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f440a8910 0x7f4f440b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.093+0000 7f4f4ac12640 1 -- 172.21.15.40:0/1131062270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f440b3ae0 msgr2=0x7f4f441d5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.093+0000 7f4f4ac12640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f440b3ae0 0x7f4f441d5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.093+0000 7f4f4ac12640 10 monclient: _finish_hunting 0 2023-07-29T13:28:19.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.093+0000 7f4f4ac12640 1 monclient: found mon.noname-c 2023-07-29T13:28:19.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.093+0000 7f4f4ac12640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:19.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.093+0000 7f4f4ac12640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:28:19.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.093+0000 7f4f4ac12640 1 -- 172.21.15.40:0/1131062270 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c009a70 con 0x7f4f44141320 2023-07-29T13:28:19.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.096+0000 7f4f4ac12640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 0x7f4f441d8f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c033000 tx=0x7f4f3c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:19.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.096+0000 7f4f3affd640 10 client.?.objecter ms_handle_connect 0x7f4f44141320 2023-07-29T13:28:19.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.096+0000 7f4f3affd640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 1 -- 172.21.15.40:0/1131062270 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c03e610 con 0x7f4f44141320 2023-07-29T13:28:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 10 monclient: dump: 2023-07-29T13:28:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.112 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.112 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.112 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 10 monclient: _finish_auth 0 2023-07-29T13:28:19.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 10 monclient: _check_auth_tickets 2023-07-29T13:28:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 1 -- 172.21.15.40:0/1131062270 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c02ce90 con 0x7f4f44141320 2023-07-29T13:28:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f4c675640 5 monclient: authenticate success, global_id 4277 2023-07-29T13:28:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f4c675640 10 monclient: _renew_subs 2023-07-29T13:28:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f4c675640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f4c675640 1 -- 172.21.15.40:0/1131062270 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f441d9460 con 0x7f4f44141320 2023-07-29T13:28:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:19.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 1 -- 172.21.15.40:0/1131062270 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c03e9d0 con 0x7f4f44141320 2023-07-29T13:28:19.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:19.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f3affd640 10 monclient: dump: 2023-07-29T13:28:19.114 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.114 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.114 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.115 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f48c0e640 4 set_mon_vals no callback set 2023-07-29T13:28:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f4c675640 10 client.4277.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f4c675640 10 monclient: _renew_subs 2023-07-29T13:28:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f4c675640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f4c675640 1 -- 172.21.15.40:0/1131062270 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f441d98e0 con 0x7f4f44141320 2023-07-29T13:28:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.101+0000 7f4f4c675640 1 librados: init done 2023-07-29T13:28:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.102+0000 7f4f4c675640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.102+0000 7f4f4c675640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.102+0000 7f4f4c675640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.102+0000 7f4f4c675640 1 -- 172.21.15.40:0/1131062270 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f441410b0 con 0x7f4f44141320 2023-07-29T13:28:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.109+0000 7f573eafd640 10 monclient: get_monmap_and_config 2023-07-29T13:28:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.109+0000 7f573eafd640 10 monclient: build_initial_monmap 2023-07-29T13:28:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.109+0000 7f573eafd640 10 monclient: monmap: 2023-07-29T13:28:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:19.110622+0000 2023-07-29T13:28:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:19.110622+0000 2023-07-29T13:28:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:19.120 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 1 Processor -- start 2023-07-29T13:28:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 1 -- start start 2023-07-29T13:28:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 10 monclient: init 2023-07-29T13:28:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2023-07-29T13:28:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57380b3ae0 0x7f573813dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 10 monclient(hunting): picked mon.noname-b con 0x7f57380b3ae0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f57380a8910 0x7f573813e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 10 monclient(hunting): picked mon.noname-c con 0x7f57380a8910 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5738140c10 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57380b3eb0 con 0x7f5738140c10 2023-07-29T13:28:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57380b0f10 con 0x7f57380a8910 2023-07-29T13:28:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57380a9950 con 0x7f57380b3ae0 2023-07-29T13:28:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.110+0000 7f573eafd640 10 monclient(hunting): authenticate will time out at 1421.243946s 2023-07-29T13:28:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.111+0000 7f573d09a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5738140c10 0x7f5738141070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51328/0 (socket says 172.21.15.40:51328) 2023-07-29T13:28:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.111+0000 7f573d09a640 1 -- 172.21.15.40:0/63335391 learned_addr learned my addr 172.21.15.40:0/63335391 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:28:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.114+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028605715 0 0) 0x7f57380b3eb0 con 0x7f5738140c10 2023-07-29T13:28:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.114+0000 7f57377fe640 10 monclient(hunting): _negotiate 2023-07-29T13:28:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.114+0000 7f57377fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.114+0000 7f57377fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.114+0000 7f57377fe640 10 monclient(hunting): my global_id is 4359 2023-07-29T13:28:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.114+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5728003520 con 0x7f5738140c10 2023-07-29T13:28:19.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3036869092 0 0) 0x7f5728003520 con 0x7f5738140c10 2023-07-29T13:28:19.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57380b3ae0 msgr2=0x7f573813dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 --2- 172.21.15.40:0/63335391 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57380b3ae0 0x7f573813dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 >> v2:172.21.15.45:3300/0 conn(0x7f57380a8910 msgr2=0x7f573813e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 --2- 172.21.15.40:0/63335391 >> v2:172.21.15.45:3300/0 conn(0x7f57380a8910 0x7f573813e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient: _finish_hunting 0 2023-07-29T13:28:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 monclient: found mon.noname-a-legacy 2023-07-29T13:28:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.40:6789/0 2023-07-29T13:28:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57380a9c30 con 0x7f5738140c10 2023-07-29T13:28:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient: _finish_auth 0 2023-07-29T13:28:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient: _check_auth_tickets 2023-07-29T13:28:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3319386150 0 0) 0x7f57300019f0 con 0x7f5738140c10 2023-07-29T13:28:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-07-29T13:28:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient: dump: 2023-07-29T13:28:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.128 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] but i'm connected to v1:172.21.15.40:6789/0, reconnecting 2023-07-29T13:28:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 >> v1:172.21.15.40:6789/0 conn(0x7f5738140c10 legacy=0x7f5738141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T13:28:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 --2- 172.21.15.40:0/63335391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 0x7f5728003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient(hunting): picked mon.a con 0x7f5728003970 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 --2- 172.21.15.40:0/63335391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57280047c0 0x7f5728004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient(hunting): picked mon.b con 0x7f57280047c0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 --2- 172.21.15.40:0/63335391 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57380b3ae0 0x7f572800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient(hunting): picked mon.c con 0x7f57380b3ae0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57380a9950 con 0x7f5728003970 2023-07-29T13:28:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57280054a0 con 0x7f57280047c0 2023-07-29T13:28:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5728005660 con 0x7f57380b3ae0 2023-07-29T13:28:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.119+0000 7f57377fe640 10 monclient(hunting): _finish_auth 0 2023-07-29T13:28:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.121+0000 7f573c899640 1 --2- 172.21.15.40:0/63335391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 0x7f5728003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.122+0000 7f573c899640 10 monclient(hunting): get_auth_request con 0x7f5728003970 auth_method 0 2023-07-29T13:28:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.122+0000 7f573c899640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.122+0000 7f573c899640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.122+0000 7f573c899640 10 monclient(hunting): _init_auth already have auth, reseting 2023-07-29T13:28:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.123+0000 7f573c899640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:19.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.123+0000 7f573c899640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:19.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.123+0000 7f573c899640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-07-29T13:28:19.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 10 monclient(hunting): handle_auth_done global_id 4359 payload 293 2023-07-29T13:28:19.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 1 -- 172.21.15.40:0/63335391 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57380b3ae0 msgr2=0x7f572800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:19.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 1 --2- 172.21.15.40:0/63335391 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57380b3ae0 0x7f572800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 1 -- 172.21.15.40:0/63335391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57280047c0 msgr2=0x7f5728004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:19.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 1 --2- 172.21.15.40:0/63335391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57280047c0 0x7f5728004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 10 monclient: _finish_hunting 0 2023-07-29T13:28:19.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 1 monclient: found mon.a 2023-07-29T13:28:19.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:19.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:19.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 1 -- 172.21.15.40:0/63335391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5728003520 con 0x7f5728003970 2023-07-29T13:28:19.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 10 monclient: _finish_auth 0 2023-07-29T13:28:19.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 10 monclient: _check_auth_tickets 2023-07-29T13:28:19.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573c899640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:19.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573eafd640 5 monclient: authenticate success, global_id 4359 2023-07-29T13:28:19.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.124+0000 7f573eafd640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:19.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.125+0000 7f4f3affd640 1 -- 172.21.15.40:0/1131062270 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f4f3c03b050 con 0x7f4f44141320 2023-07-29T13:28:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.126+0000 7f4f3affd640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f4f2806bfe0 0x7f4f2806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.126+0000 7f4f4a411640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f4f2806bfe0 0x7f4f2806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.126+0000 7f4f4a411640 10 monclient: get_auth_request con 0x7f4f2806bfe0 auth_method 0 2023-07-29T13:28:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.126+0000 7f4f3affd640 10 client.4277.objecter ms_handle_connect 0x7f4f2806bfe0 2023-07-29T13:28:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.126+0000 7f4f4a411640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f4f2806bfe0 0x7f4f2806e490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009710 tx=0x7f4f34006c90 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:19.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.127+0000 7f573c899640 1 --2- 172.21.15.40:0/63335391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 0x7f5728003d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f572c009860 tx=0x7f57280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6a1d1f6d974f45 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:19.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c032600 con 0x7f5728003970 2023-07-29T13:28:19.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f57377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f57377fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f57377fe640 10 monclient: dump: 2023-07-29T13:28:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.143 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f573eafd640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c02e050 con 0x7f5728003970 2023-07-29T13:28:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f57377fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f573eafd640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f573eafd640 4 set_mon_vals no callback set 2023-07-29T13:28:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f57377fe640 1 -- 172.21.15.40:0/63335391 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c032a00 con 0x7f5728003970 2023-07-29T13:28:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f57377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f57377fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.131+0000 7f57377fe640 10 monclient: dump: 2023-07-29T13:28:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.146 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.132+0000 7f573eafd640 10 monclient: shutdown 2023-07-29T13:28:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.132+0000 7f573eafd640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.132+0000 7f573eafd640 1 -- 172.21.15.40:0/63335391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 msgr2=0x7f5728003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.132+0000 7f573eafd640 1 --2- 172.21.15.40:0/63335391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 0x7f5728003d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f572c009860 tx=0x7f57280057f0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.132+0000 7f573d09a640 1 -- 172.21.15.40:0/63335391 reap_dead start 2023-07-29T13:28:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.132+0000 7f573eafd640 1 -- 172.21.15.40:0/63335391 shutdown_connections 2023-07-29T13:28:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.132+0000 7f573eafd640 1 -- 172.21.15.40:0/63335391 >> 172.21.15.40:0/63335391 conn(0x7f5738070fc0 msgr2=0x7f57380713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.132+0000 7f573eafd640 1 -- 172.21.15.40:0/63335391 shutdown_connections 2023-07-29T13:28:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.132+0000 7f573eafd640 1 -- 172.21.15.40:0/63335391 wait complete. 2023-07-29T13:28:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.133+0000 7f573eafd640 10 monclient: build_initial_monmap 2023-07-29T13:28:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.133+0000 7f573eafd640 10 monclient: monmap: 2023-07-29T13:28:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:19.149 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:19.149 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.133+0000 7f573eafd640 1 librados: starting msgr at 2023-07-29T13:28:19.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.133+0000 7f573eafd640 1 librados: starting objecter 2023-07-29T13:28:19.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.133+0000 7f573eafd640 1 Processor -- start 2023-07-29T13:28:19.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.133+0000 7f573eafd640 1 -- start start 2023-07-29T13:28:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.133+0000 7f573eafd640 1 librados: setting wanted keys 2023-07-29T13:28:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.133+0000 7f573eafd640 1 librados: calling monclient init 2023-07-29T13:28:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.133+0000 7f573eafd640 10 monclient: init 2023-07-29T13:28:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T13:28:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 0x7f57381d9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 10 monclient(hunting): picked mon.noname-a con 0x7f5728003970 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57380ab0a0 0x7f57380abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 10 monclient(hunting): picked mon.noname-c con 0x7f57380ab0a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5738143840 0x7f57380ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 10 monclient(hunting): picked mon.noname-b con 0x7f5738143840 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57380ac710 con 0x7f5728003970 2023-07-29T13:28:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57381d8df0 con 0x7f5738143840 2023-07-29T13:28:19.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57381d9850 con 0x7f57380ab0a0 2023-07-29T13:28:19.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:19.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.134+0000 7f573eafd640 10 monclient(hunting): authenticate will time out at 1421.267946s 2023-07-29T13:28:19.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.135+0000 7f573c899640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 0x7f57381d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.135+0000 7f573c899640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 0x7f57381d9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60736/0 (socket says 172.21.15.40:60736) 2023-07-29T13:28:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.135+0000 7f573c899640 1 -- 172.21.15.40:0/3747094394 learned_addr learned my addr 172.21.15.40:0/3747094394 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.135+0000 7f573c899640 10 monclient(hunting): get_auth_request con 0x7f5728003970 auth_method 0 2023-07-29T13:28:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.135+0000 7f573c899640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.135+0000 7f573c899640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.135+0000 7f573c899640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.136+0000 7f573c899640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:19.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.136+0000 7f573c899640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:19.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.136+0000 7f573c899640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:19.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.139+0000 7f573c899640 10 monclient(hunting): handle_auth_done global_id 4362 payload 824 2023-07-29T13:28:19.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.139+0000 7f573c899640 1 -- 172.21.15.40:0/3747094394 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57380ab0a0 msgr2=0x7f57380abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.139+0000 7f573c899640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57380ab0a0 0x7f57380abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.139+0000 7f573c899640 1 -- 172.21.15.40:0/3747094394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5738143840 msgr2=0x7f57380ac1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.139+0000 7f573c899640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5738143840 0x7f57380ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.139+0000 7f573c899640 10 monclient: _finish_hunting 0 2023-07-29T13:28:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.139+0000 7f573c899640 1 monclient: found mon.noname-a 2023-07-29T13:28:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.139+0000 7f573c899640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.139+0000 7f573c899640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.139+0000 7f573c899640 1 -- 172.21.15.40:0/3747094394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57381d9a10 con 0x7f5728003970 2023-07-29T13:28:19.155 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-29T13:28:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.141+0000 7f573c899640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 0x7f57381d9310 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f572c000c00 tx=0x7f572c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.141+0000 7f57357fa640 10 client.?.objecter ms_handle_connect 0x7f5728003970 2023-07-29T13:28:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.141+0000 7f57357fa640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 1 -- 172.21.15.40:0/3747094394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c03e430 con 0x7f5728003970 2023-07-29T13:28:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 10 monclient: dump: 2023-07-29T13:28:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.159 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 10 monclient: _finish_auth 0 2023-07-29T13:28:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 10 monclient: _check_auth_tickets 2023-07-29T13:28:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 1 -- 172.21.15.40:0/3747094394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c03e590 con 0x7f5728003970 2023-07-29T13:28:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 1 -- 172.21.15.40:0/3747094394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c03e920 con 0x7f5728003970 2023-07-29T13:28:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f573eafd640 5 monclient: authenticate success, global_id 4362 2023-07-29T13:28:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f573eafd640 10 monclient: _renew_subs 2023-07-29T13:28:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f573eafd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f573eafd640 1 -- 172.21.15.40:0/3747094394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5738139020 con 0x7f5728003970 2023-07-29T13:28:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:19.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f57357fa640 10 monclient: dump: 2023-07-29T13:28:19.162 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.162 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.162 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.164 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f5736ffd640 4 set_mon_vals no callback set 2023-07-29T13:28:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f573eafd640 10 client.4362.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f573eafd640 10 monclient: _renew_subs 2023-07-29T13:28:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f573eafd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f573eafd640 1 -- 172.21.15.40:0/3747094394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5738139530 con 0x7f5728003970 2023-07-29T13:28:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.145+0000 7f573eafd640 1 librados: init done 2023-07-29T13:28:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.146+0000 7f573eafd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.146+0000 7f573eafd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:19.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.146+0000 7f573eafd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:19.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.146+0000 7f573eafd640 1 -- 172.21.15.40:0/3747094394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57381d8fb0 con 0x7f5728003970 2023-07-29T13:28:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.171+0000 7f57357fa640 1 -- 172.21.15.40:0/3747094394 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f572c03ea80 con 0x7f5728003970 2023-07-29T13:28:19.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.171+0000 7f57357fa640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f571c06bf60 0x7f571c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.171+0000 7f5737fff640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f571c06bf60 0x7f571c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:19.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.172+0000 7f5737fff640 10 monclient: get_auth_request con 0x7f571c06bf60 auth_method 0 2023-07-29T13:28:19.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.172+0000 7f5737fff640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f571c06bf60 0x7f571c06e410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f57380b5fd0 tx=0x7f5724006d70 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:19.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.172+0000 7f57357fa640 10 client.4362.objecter ms_handle_connect 0x7f571c06bf60 2023-07-29T13:28:19.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.174+0000 7f57357fa640 1 -- 172.21.15.40:0/3747094394 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f572c07dde0 con 0x7f5728003970 2023-07-29T13:28:19.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.174+0000 7f57357fa640 10 client.4362.objecter ms_dispatch 0x7f5738000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:19.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.174+0000 7f57357fa640 3 client.4362.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:19.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.174+0000 7f57357fa640 3 client.4362.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:19.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.174+0000 7f57357fa640 20 client.4362.objecter dump_active .. 0 homeless 2023-07-29T13:28:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.253+0000 7f1cacff9640 1 -- 172.21.15.40:0/277076583 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1c9805b510 con 0x7f1cb00b3ae0 2023-07-29T13:28:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.253+0000 7f1cacff9640 10 client.4318.objecter ms_dispatch 0x7f1cb0000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.253+0000 7f1cacff9640 3 client.4318.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.253+0000 7f1cacff9640 3 client.4318.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.253+0000 7f1cacff9640 20 client.4318.objecter dump_active .. 0 homeless 2023-07-29T13:28:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.264+0000 7f1cacff9640 1 -- 172.21.15.40:0/277076583 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f1c98053940 con 0x7f1cb00b3ae0 2023-07-29T13:28:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.264+0000 7f1cacff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.264+0000 7f1cacff9640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.278+0000 7f1acf02f640 10 monclient: get_monmap_and_config 2023-07-29T13:28:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.278+0000 7f1acf02f640 10 monclient: build_initial_monmap 2023-07-29T13:28:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.278+0000 7f1acf02f640 10 monclient: monmap: 2023-07-29T13:28:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:19.279825+0000 2023-07-29T13:28:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:19.279825+0000 2023-07-29T13:28:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:19.282 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.279+0000 7f1acf02f640 1 Processor -- start 2023-07-29T13:28:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.279+0000 7f1acf02f640 1 -- start start 2023-07-29T13:28:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.279+0000 7f1acf02f640 10 monclient: init 2023-07-29T13:28:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.279+0000 7f1acf02f640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1acf02f640 10 monclient: _add_conns ranks=[0,4,1,3,2] 2023-07-29T13:28:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1acf02f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1ac80b74a0 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1acf02f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ac813e4f0 0x7f1ac80aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1acf02f640 10 monclient(hunting): picked mon.noname-c con 0x7f1ac813e4f0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1acf02f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1ac81445e0 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1acf02f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac80b7870 con 0x7f1ac80b74a0 2023-07-29T13:28:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1acf02f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac80a91b0 con 0x7f1ac81445e0 2023-07-29T13:28:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1acf02f640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1acf02f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac80a9490 con 0x7f1ac813e4f0 2023-07-29T13:28:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1acf02f640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1acf02f640 10 monclient(hunting): authenticate will time out at 1421.413270s 2023-07-29T13:28:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.280+0000 7f1ac7fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ac813e4f0 0x7f1ac80aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.281+0000 7f1acd5cc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1ac81445e0 0x7f1ac8144a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.40:46346/0 (socket says 172.21.15.40:46346) 2023-07-29T13:28:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.281+0000 7f1acd5cc640 1 -- 172.21.15.40:0/7042217 learned_addr learned my addr 172.21.15.40:0/7042217 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:28:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.281+0000 7f1ac7fff640 10 monclient(hunting): get_auth_request con 0x7f1ac813e4f0 auth_method 0 2023-07-29T13:28:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.281+0000 7f1ac7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.281+0000 7f1ac7fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.281+0000 7f1ac7fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.281+0000 7f1ac7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.281+0000 7f1ac7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.281+0000 7f1ac7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.283+0000 7f1ac7fff640 10 monclient(hunting): handle_auth_done global_id 4321 payload 274 2023-07-29T13:28:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.283+0000 7f1ac7fff640 1 -- 172.21.15.40:0/7042217 >> v1:172.21.15.45:6789/0 conn(0x7f1ac81445e0 legacy=0x7f1ac8144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.283+0000 7f1ac7fff640 1 -- 172.21.15.40:0/7042217 >> v1:172.21.15.40:6789/0 conn(0x7f1ac80b74a0 legacy=0x7f1ac813dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.283+0000 7f1ac7fff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.283+0000 7f1ac7fff640 1 monclient: found mon.noname-c 2023-07-29T13:28:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.283+0000 7f1ac7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.283+0000 7f1ac7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:28:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.283+0000 7f1ac7fff640 1 -- 172.21.15.40:0/7042217 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac80a9770 con 0x7f1ac813e4f0 2023-07-29T13:28:19.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.288+0000 7f1ac7fff640 1 --2- 172.21.15.40:0/7042217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac813e4f0 0x7f1ac80aa150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40097a0 tx=0x7f1ab402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5d114614be1347f server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 1 -- 172.21.15.40:0/7042217 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab402e070 con 0x7f1ac813e4f0 2023-07-29T13:28:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 10 monclient: dump: 2023-07-29T13:28:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.295 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 10 monclient: _finish_auth 0 2023-07-29T13:28:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 10 monclient: _check_auth_tickets 2023-07-29T13:28:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 1 -- 172.21.15.40:0/7042217 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab40315a0 con 0x7f1ac813e4f0 2023-07-29T13:28:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1acf02f640 5 monclient: authenticate success, global_id 4321 2023-07-29T13:28:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1acf02f640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1acf02f640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1acf02f640 4 set_mon_vals no callback set 2023-07-29T13:28:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 1 -- 172.21.15.40:0/7042217 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4031930 con 0x7f1ac813e4f0 2023-07-29T13:28:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.292+0000 7f1ac77fe640 10 monclient: dump: 2023-07-29T13:28:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.299 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.299 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.299 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.293+0000 7f1acf02f640 10 monclient: shutdown 2023-07-29T13:28:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.293+0000 7f1acf02f640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.293+0000 7f1acf02f640 1 -- 172.21.15.40:0/7042217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac813e4f0 msgr2=0x7f1ac80aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.293+0000 7f1acf02f640 1 --2- 172.21.15.40:0/7042217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac813e4f0 0x7f1ac80aa150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40097a0 tx=0x7f1ab402b840 comp rx=0 tx=0).stop 2023-07-29T13:28:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.293+0000 7f1acf02f640 1 -- 172.21.15.40:0/7042217 shutdown_connections 2023-07-29T13:28:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.293+0000 7f1acf02f640 1 --2- 172.21.15.40:0/7042217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac813e4f0 0x7f1ac80aa150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.293+0000 7f1acf02f640 1 -- 172.21.15.40:0/7042217 >> 172.21.15.40:0/7042217 conn(0x7f1ac8065b00 msgr2=0x7f1ac8065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.293+0000 7f1acf02f640 1 -- 172.21.15.40:0/7042217 shutdown_connections 2023-07-29T13:28:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.293+0000 7f1acf02f640 1 -- 172.21.15.40:0/7042217 wait complete. 2023-07-29T13:28:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.294+0000 7f1acf02f640 10 monclient: build_initial_monmap 2023-07-29T13:28:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.294+0000 7f1acf02f640 10 monclient: monmap: 2023-07-29T13:28:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:19.303 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.294+0000 7f1acf02f640 1 librados: starting msgr at 2023-07-29T13:28:19.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.294+0000 7f1acf02f640 1 librados: starting objecter 2023-07-29T13:28:19.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.294+0000 7f1acf02f640 1 Processor -- start 2023-07-29T13:28:19.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.294+0000 7f1acf02f640 1 -- start start 2023-07-29T13:28:19.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.294+0000 7f1acf02f640 1 librados: setting wanted keys 2023-07-29T13:28:19.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.294+0000 7f1acf02f640 1 librados: calling monclient init 2023-07-29T13:28:19.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.294+0000 7f1acf02f640 10 monclient: init 2023-07-29T13:28:19.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T13:28:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1ac80b74a0 0x7f1ac8143840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 10 monclient(hunting): picked mon.noname-c con 0x7f1ac80b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac813e4f0 0x7f1ac81373e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 10 monclient(hunting): picked mon.noname-a con 0x7f1ac813e4f0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac81445e0 0x7f1ac8137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 10 monclient(hunting): picked mon.noname-b con 0x7f1ac81445e0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80ab240 con 0x7f1ac813e4f0 2023-07-29T13:28:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80ac620 con 0x7f1ac81445e0 2023-07-29T13:28:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80aad20 con 0x7f1ac80b74a0 2023-07-29T13:28:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.295+0000 7f1acf02f640 10 monclient(hunting): authenticate will time out at 1421.429056s 2023-07-29T13:28:19.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.296+0000 7f1acd5cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac81445e0 0x7f1ac8137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:19.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.296+0000 7f1acd5cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac81445e0 0x7f1ac8137920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:42684/0 (socket says 172.21.15.40:42684) 2023-07-29T13:28:19.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.296+0000 7f1acd5cc640 1 -- 172.21.15.40:0/1814203736 learned_addr learned my addr 172.21.15.40:0/1814203736 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:19.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.296+0000 7f1acd5cc640 10 monclient(hunting): get_auth_request con 0x7f1ac81445e0 auth_method 0 2023-07-29T13:28:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.296+0000 7f1acd5cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.296+0000 7f1acd5cc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.296+0000 7f1acd5cc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.297+0000 7f1acd5cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.297+0000 7f1acd5cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.297+0000 7f1acd5cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:19.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.300+0000 7f1acd5cc640 10 monclient(hunting): handle_auth_done global_id 4327 payload 824 2023-07-29T13:28:19.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.300+0000 7f1acd5cc640 1 -- 172.21.15.40:0/1814203736 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1ac80b74a0 msgr2=0x7f1ac8143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.300+0000 7f1acd5cc640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1ac80b74a0 0x7f1ac8143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.300+0000 7f1acd5cc640 1 -- 172.21.15.40:0/1814203736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac813e4f0 msgr2=0x7f1ac81373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.300+0000 7f1acd5cc640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac813e4f0 0x7f1ac81373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.300+0000 7f1acd5cc640 10 monclient: _finish_hunting 0 2023-07-29T13:28:19.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.300+0000 7f1acd5cc640 1 monclient: found mon.noname-b 2023-07-29T13:28:19.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.300+0000 7f1acd5cc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:19.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.300+0000 7f1acd5cc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:19.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.300+0000 7f1acd5cc640 1 -- 172.21.15.40:0/1814203736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab4009400 con 0x7f1ac81445e0 2023-07-29T13:28:19.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.302+0000 7f1acd5cc640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac81445e0 0x7f1ac8137920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000cab0 tx=0x7f1ac000cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:19.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.302+0000 7f1ac57fa640 10 client.?.objecter ms_handle_connect 0x7f1ac81445e0 2023-07-29T13:28:19.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.302+0000 7f1ac57fa640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:19.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 1 -- 172.21.15.40:0/1814203736 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac000ac30 con 0x7f1ac81445e0 2023-07-29T13:28:19.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:19.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 10 monclient: dump: 2023-07-29T13:28:19.312 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.313 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.313 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.313 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.313 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.313 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.313 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.313 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.313 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.314 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 10 monclient: _finish_auth 0 2023-07-29T13:28:19.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 10 monclient: _check_auth_tickets 2023-07-29T13:28:19.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:19.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 1 -- 172.21.15.40:0/1814203736 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac000ad90 con 0x7f1ac81445e0 2023-07-29T13:28:19.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:19.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1acf02f640 5 monclient: authenticate success, global_id 4327 2023-07-29T13:28:19.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 1 -- 172.21.15.40:0/1814203736 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0012600 con 0x7f1ac81445e0 2023-07-29T13:28:19.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:19.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:19.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac57fa640 10 monclient: dump: 2023-07-29T13:28:19.315 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:19.315 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:19.315 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.315 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:19.315 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:19.316 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:19.316 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:19.316 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:19.316 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:19.316 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:19.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1acf02f640 10 monclient: _renew_subs 2023-07-29T13:28:19.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1acf02f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:19.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1ac6ffd640 4 set_mon_vals no callback set 2023-07-29T13:28:19.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1acf02f640 1 -- 172.21.15.40:0/1814203736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac8137ec0 con 0x7f1ac81445e0 2023-07-29T13:28:19.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1acf02f640 10 client.4327.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:19.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1acf02f640 10 monclient: _renew_subs 2023-07-29T13:28:19.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1acf02f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:19.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1acf02f640 1 -- 172.21.15.40:0/1814203736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac80b36f0 con 0x7f1ac81445e0 2023-07-29T13:28:19.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.306+0000 7f1acf02f640 1 librados: init done 2023-07-29T13:28:19.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.308+0000 7f1acf02f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:19.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.308+0000 7f1acf02f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:19.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.308+0000 7f1acf02f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:19.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.308+0000 7f1acf02f640 1 -- 172.21.15.40:0/1814203736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac80aaf40 con 0x7f1ac81445e0 2023-07-29T13:28:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.327+0000 7f1ac57fa640 1 -- 172.21.15.40:0/1814203736 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f1ac0007070 con 0x7f1ac81445e0 2023-07-29T13:28:19.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.327+0000 7f1ac57fa640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1aac06bfb0 0x7f1aac06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:19.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.328+0000 7f1accdcb640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1aac06bfb0 0x7f1aac06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.328+0000 7f1accdcb640 10 monclient: get_auth_request con 0x7f1aac06bfb0 auth_method 0 2023-07-29T13:28:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.328+0000 7f1accdcb640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1aac06bfb0 0x7f1aac06e460 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1abc001550 tx=0x7f1abc007990 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.328+0000 7f1ac57fa640 10 client.4327.objecter ms_handle_connect 0x7f1aac06bfb0 2023-07-29T13:28:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.330+0000 7f1ac57fa640 1 -- 172.21.15.40:0/1814203736 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1ac005bd00 con 0x7f1ac81445e0 2023-07-29T13:28:19.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.330+0000 7f1ac57fa640 10 client.4327.objecter ms_dispatch 0x7f1ac8000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:19.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.330+0000 7f1ac57fa640 3 client.4327.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:19.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.330+0000 7f1ac57fa640 3 client.4327.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:19.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.330+0000 7f1ac57fa640 20 client.4327.objecter dump_active .. 0 homeless 2023-07-29T13:28:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.439+0000 7f4f3affd640 1 -- 172.21.15.40:0/1131062270 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4f3c07e640 con 0x7f4f44141320 2023-07-29T13:28:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.439+0000 7f4f3affd640 10 client.4277.objecter ms_dispatch 0x7f4f44000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:19.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.439+0000 7f4f3affd640 3 client.4277.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:19.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.439+0000 7f4f3affd640 3 client.4277.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:19.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.439+0000 7f4f3affd640 20 client.4277.objecter dump_active .. 0 homeless 2023-07-29T13:28:19.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.439+0000 7f4f3affd640 1 -- 172.21.15.40:0/1131062270 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f4f3c076a70 con 0x7f4f44141320 2023-07-29T13:28:19.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.440+0000 7f4f3affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:19.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.440+0000 7f4f3affd640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.470+0000 7f1cb587b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2023-07-29T13:28:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.470+0000 7f1cb587b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-07-29T13:28:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.470+0000 7f1cb587b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.470+0000 7f1cb587b640 1 -- 172.21.15.40:0/277076583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1cb00b5bf0 con 0x7f1cb00b3ae0 2023-07-29T13:28:19.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.488+0000 7f57357fa640 1 -- 172.21.15.40:0/3747094394 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f572c076210 con 0x7f5728003970 2023-07-29T13:28:19.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.488+0000 7f57357fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:19.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.488+0000 7f57357fa640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.580+0000 7f1ac57fa640 1 -- 172.21.15.40:0/1814203736 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f1ac0054130 con 0x7f1ac81445e0 2023-07-29T13:28:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.580+0000 7f1ac57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.580+0000 7f1ac57fa640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.591+0000 7f1cacff9640 1 -- 172.21.15.40:0/277076583 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1c98017020 con 0x7f1cb00b3ae0 2023-07-29T13:28:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.591+0000 7f1cacff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-07-29T13:28:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.591+0000 7f1cacff9640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:19.593 INFO:teuthology.orchestra.run.smithi040.stdout:73014444038 2023-07-29T13:28:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 10 librados: watch_flush enter 2023-07-29T13:28:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 10 librados: watch_flush exit 2023-07-29T13:28:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 20 client.4318.objecter shutdown clearing up homeless session... 2023-07-29T13:28:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 10 client.4318.objecter successfully canceled tick 2023-07-29T13:28:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 -- 172.21.15.40:0/277076583 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1c9006bfe0 msgr2=0x7f1c9006e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1c9006bfe0 0x7f1c9006e490 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00090b0 tx=0x7f1ca003d040 comp rx=0 tx=0).stop 2023-07-29T13:28:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 10 monclient: shutdown 2023-07-29T13:28:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 -- 172.21.15.40:0/277076583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb00b3ae0 msgr2=0x7f1cb00af310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb00b3ae0 0x7f1cb00af310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c410 tx=0x7f1c9800c810 comp rx=0 tx=0).stop 2023-07-29T13:28:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 -- 172.21.15.40:0/277076583 shutdown_connections 2023-07-29T13:28:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1c9006bfe0 0x7f1c9006e490 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb00b3ae0 0x7f1cb00af310 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb0141320 0x7f1cb00af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 --2- 172.21.15.40:0/277076583 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1cb00a8910 0x7f1cb01d60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 -- 172.21.15.40:0/277076583 >> 172.21.15.40:0/277076583 conn(0x7f1cb0070fc0 msgr2=0x7f1cb00b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 -- 172.21.15.40:0/277076583 shutdown_connections 2023-07-29T13:28:19.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 -- 172.21.15.40:0/277076583 wait complete. 2023-07-29T13:28:19.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.594+0000 7f1cb587b640 1 librados: shutdown 2023-07-29T13:28:19.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444038 got 73014444038 for osd.1 2023-07-29T13:28:19.611 DEBUG:teuthology.parallel:result is None 2023-07-29T13:28:19.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.638+0000 7f4f4c675640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2023-07-29T13:28:19.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.638+0000 7f4f4c675640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-07-29T13:28:19.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.638+0000 7f4f4c675640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:19.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.638+0000 7f4f4c675640 1 -- 172.21.15.40:0/1131062270 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4f44141800 con 0x7f4f44141320 2023-07-29T13:28:19.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.642+0000 7f4f3affd640 1 -- 172.21.15.40:0/1131062270 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4f3c04e090 con 0x7f4f44141320 2023-07-29T13:28:19.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.642+0000 7f4f3affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-07-29T13:28:19.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.642+0000 7f4f3affd640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:19.645 INFO:teuthology.orchestra.run.smithi040.stdout:73014444037 2023-07-29T13:28:19.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.644+0000 7f4f4c675640 10 librados: watch_flush enter 2023-07-29T13:28:19.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.644+0000 7f4f4c675640 10 librados: watch_flush exit 2023-07-29T13:28:19.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.644+0000 7f4f4c675640 20 client.4277.objecter shutdown clearing up homeless session... 2023-07-29T13:28:19.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.644+0000 7f4f4c675640 10 client.4277.objecter successfully canceled tick 2023-07-29T13:28:19.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.644+0000 7f4f4c675640 1 -- 172.21.15.40:0/1131062270 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f4f2806bfe0 msgr2=0x7f4f2806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f4f2806bfe0 0x7f4f2806e490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009710 tx=0x7f4f34006c90 comp rx=0 tx=0).stop 2023-07-29T13:28:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 10 monclient: shutdown 2023-07-29T13:28:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 -- 172.21.15.40:0/1131062270 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 msgr2=0x7f4f441d8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 0x7f4f441d8f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c033000 tx=0x7f4f3c004a40 comp rx=0 tx=0).stop 2023-07-29T13:28:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 -- 172.21.15.40:0/1131062270 shutdown_connections 2023-07-29T13:28:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f4f2806bfe0 0x7f4f2806e490 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f44141320 0x7f4f441d8f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f440b3ae0 0x7f4f441d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 --2- 172.21.15.40:0/1131062270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f440a8910 0x7f4f440b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 -- 172.21.15.40:0/1131062270 >> 172.21.15.40:0/1131062270 conn(0x7f4f44070fc0 msgr2=0x7f4f4413c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 -- 172.21.15.40:0/1131062270 shutdown_connections 2023-07-29T13:28:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 -- 172.21.15.40:0/1131062270 wait complete. 2023-07-29T13:28:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.645+0000 7f4f4c675640 1 librados: shutdown 2023-07-29T13:28:19.659 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444037 got 73014444037 for osd.2 2023-07-29T13:28:19.659 DEBUG:teuthology.parallel:result is None 2023-07-29T13:28:19.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.673+0000 7f573eafd640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2023-07-29T13:28:19.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.673+0000 7f573eafd640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-07-29T13:28:19.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.673+0000 7f573eafd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T13:28:19.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.673+0000 7f573eafd640 1 -- 172.21.15.40:0/3747094394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f57380ac710 con 0x7f5728003970 2023-07-29T13:28:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.677+0000 7f57357fa640 1 -- 172.21.15.40:0/3747094394 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f57380ac710 con 0x7f5728003970 2023-07-29T13:28:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.677+0000 7f57357fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-07-29T13:28:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.677+0000 7f57357fa640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:19.679 INFO:teuthology.orchestra.run.smithi040.stdout:64424509446 2023-07-29T13:28:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 10 librados: watch_flush enter 2023-07-29T13:28:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 10 librados: watch_flush exit 2023-07-29T13:28:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 20 client.4362.objecter shutdown clearing up homeless session... 2023-07-29T13:28:19.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 10 client.4362.objecter successfully canceled tick 2023-07-29T13:28:19.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 -- 172.21.15.40:0/3747094394 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f571c06bf60 msgr2=0x7f571c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f571c06bf60 0x7f571c06e410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f57380b5fd0 tx=0x7f5724006d70 comp rx=0 tx=0).stop 2023-07-29T13:28:19.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 10 monclient: shutdown 2023-07-29T13:28:19.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:19.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 -- 172.21.15.40:0/3747094394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 msgr2=0x7f57381d9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 0x7f57381d9310 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f572c000c00 tx=0x7f572c0047d0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 -- 172.21.15.40:0/3747094394 shutdown_connections 2023-07-29T13:28:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f571c06bf60 0x7f571c06e410 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5738143840 0x7f57380ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57380ab0a0 0x7f57380abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 --2- 172.21.15.40:0/3747094394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5728003970 0x7f57381d9310 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 -- 172.21.15.40:0/3747094394 >> 172.21.15.40:0/3747094394 conn(0x7f5738070fc0 msgr2=0x7f57380b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 -- 172.21.15.40:0/3747094394 shutdown_connections 2023-07-29T13:28:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 -- 172.21.15.40:0/3747094394 wait complete. 2023-07-29T13:28:19.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.679+0000 7f573eafd640 1 librados: shutdown 2023-07-29T13:28:19.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2023-07-29T13:28:19.692 DEBUG:teuthology.parallel:result is None 2023-07-29T13:28:19.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.801+0000 7f1acf02f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2023-07-29T13:28:19.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.801+0000 7f1acf02f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-07-29T13:28:19.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.801+0000 7f1acf02f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:19.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.801+0000 7f1acf02f640 1 -- 172.21.15.40:0/1814203736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1ac814c690 con 0x7f1ac81445e0 2023-07-29T13:28:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.805+0000 7f1ac57fa640 1 -- 172.21.15.40:0/1814203736 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ac0021090 con 0x7f1ac81445e0 2023-07-29T13:28:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.805+0000 7f1ac57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-07-29T13:28:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.805+0000 7f1ac57fa640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:19.807 INFO:teuthology.orchestra.run.smithi040.stdout:68719476739 2023-07-29T13:28:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 10 librados: watch_flush enter 2023-07-29T13:28:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 10 librados: watch_flush exit 2023-07-29T13:28:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 20 client.4327.objecter shutdown clearing up homeless session... 2023-07-29T13:28:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 10 client.4327.objecter successfully canceled tick 2023-07-29T13:28:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 -- 172.21.15.40:0/1814203736 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1aac06bfb0 msgr2=0x7f1aac06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1aac06bfb0 0x7f1aac06e460 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1abc001550 tx=0x7f1abc007990 comp rx=0 tx=0).stop 2023-07-29T13:28:19.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 10 monclient: shutdown 2023-07-29T13:28:19.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:19.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 -- 172.21.15.40:0/1814203736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac81445e0 msgr2=0x7f1ac8137920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:19.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac81445e0 0x7f1ac8137920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000cab0 tx=0x7f1ac000cf70 comp rx=0 tx=0).stop 2023-07-29T13:28:19.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 -- 172.21.15.40:0/1814203736 shutdown_connections 2023-07-29T13:28:19.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f1aac06bfb0 0x7f1aac06e460 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac81445e0 0x7f1ac8137920 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac813e4f0 0x7f1ac81373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 --2- 172.21.15.40:0/1814203736 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f1ac80b74a0 0x7f1ac8143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:19.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 -- 172.21.15.40:0/1814203736 >> 172.21.15.40:0/1814203736 conn(0x7f1ac8065b00 msgr2=0x7f1ac80b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:19.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 -- 172.21.15.40:0/1814203736 shutdown_connections 2023-07-29T13:28:19.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 -- 172.21.15.40:0/1814203736 wait complete. 2023-07-29T13:28:19.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:19.807+0000 7f1acf02f640 1 librados: shutdown 2023-07-29T13:28:19.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476739 for osd.4 2023-07-29T13:28:20.821 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-29T13:28:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.931+0000 7f334a23a640 10 monclient: get_monmap_and_config 2023-07-29T13:28:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.931+0000 7f334a23a640 10 monclient: build_initial_monmap 2023-07-29T13:28:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.931+0000 7f334a23a640 10 monclient: monmap: 2023-07-29T13:28:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:20.932846+0000 2023-07-29T13:28:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:20.932846+0000 2023-07-29T13:28:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:20.935 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.932+0000 7f334a23a640 1 Processor -- start 2023-07-29T13:28:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.932+0000 7f334a23a640 1 -- start start 2023-07-29T13:28:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.932+0000 7f334a23a640 10 monclient: init 2023-07-29T13:28:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2023-07-29T13:28:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f33440b3ae0 0x7f334413dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 10 monclient(hunting): picked mon.noname-c con 0x7f33440b3ae0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f33440a8910 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3344141320 0x7f334413e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 10 monclient(hunting): picked mon.noname-a con 0x7f3344141320 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33440b3eb0 con 0x7f33440a8910 2023-07-29T13:28:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33440b0f10 con 0x7f3344141320 2023-07-29T13:28:20.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:20.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33440a9950 con 0x7f33440b3ae0 2023-07-29T13:28:20.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:20.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f334a23a640 10 monclient(hunting): authenticate will time out at 1423.066336s 2023-07-29T13:28:20.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.933+0000 7f3343fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3344141320 0x7f334413e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:20.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.934+0000 7f33437fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f33440b3ae0 0x7f334413dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:20.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.934+0000 7f3343fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3344141320 0x7f334413e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33108/0 (socket says 172.21.15.40:33108) 2023-07-29T13:28:20.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.934+0000 7f3343fff640 1 -- 172.21.15.40:0/99038292 learned_addr learned my addr 172.21.15.40:0/99038292 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:20.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.934+0000 7f3343fff640 10 monclient(hunting): get_auth_request con 0x7f3344141320 auth_method 0 2023-07-29T13:28:20.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.934+0000 7f3343fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:20.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.934+0000 7f3343fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:20.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.934+0000 7f3343fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:20.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.935+0000 7f33437fe640 10 monclient(hunting): get_auth_request con 0x7f33440b3ae0 auth_method 0 2023-07-29T13:28:20.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.935+0000 7f33437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:20.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.935+0000 7f33437fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:20.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.935+0000 7f33437fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:20.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.935+0000 7f3343fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:20.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.935+0000 7f3343fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:20.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.935+0000 7f3343fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:20.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.935+0000 7f33437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:20.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.935+0000 7f33437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:20.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.935+0000 7f33437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.936+0000 7f3343fff640 10 monclient(hunting): handle_auth_done global_id 4368 payload 274 2023-07-29T13:28:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.936+0000 7f3343fff640 1 -- 172.21.15.40:0/99038292 >> v2:172.21.15.45:3300/0 conn(0x7f33440b3ae0 msgr2=0x7f334413dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.936+0000 7f3343fff640 1 --2- 172.21.15.40:0/99038292 >> v2:172.21.15.45:3300/0 conn(0x7f33440b3ae0 0x7f334413dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.936+0000 7f3343fff640 1 -- 172.21.15.40:0/99038292 >> v1:172.21.15.45:6789/0 conn(0x7f33440a8910 legacy=0x7f3344140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.936+0000 7f3343fff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.936+0000 7f3343fff640 1 monclient: found mon.noname-a 2023-07-29T13:28:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.936+0000 7f3343fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.936+0000 7f3343fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.936+0000 7f3343fff640 1 -- 172.21.15.40:0/99038292 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33440a9c30 con 0x7f3344141320 2023-07-29T13:28:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.939+0000 7f3343fff640 1 --2- 172.21.15.40:0/99038292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3344141320 0x7f334413e1d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3344066830 tx=0x7f333c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306b376adc5a7ae5 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:20.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 1 -- 172.21.15.40:0/99038292 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c02ca80 con 0x7f3344141320 2023-07-29T13:28:20.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:20.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:20.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 10 monclient: dump: 2023-07-29T13:28:20.955 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:20.957 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:20.957 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:20.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 10 monclient: _finish_auth 0 2023-07-29T13:28:20.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 10 monclient: _check_auth_tickets 2023-07-29T13:28:20.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:20.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 1 -- 172.21.15.40:0/99038292 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c02cbe0 con 0x7f3344141320 2023-07-29T13:28:20.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f334a23a640 5 monclient: authenticate success, global_id 4368 2023-07-29T13:28:20.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f334a23a640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f334a23a640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f334a23a640 4 set_mon_vals no callback set 2023-07-29T13:28:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 1 -- 172.21.15.40:0/99038292 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c0343f0 con 0x7f3344141320 2023-07-29T13:28:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.953+0000 7f33427fc640 10 monclient: dump: 2023-07-29T13:28:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:20.960 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:20.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.954+0000 7f334a23a640 10 monclient: shutdown 2023-07-29T13:28:20.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.954+0000 7f334a23a640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:20.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.954+0000 7f334a23a640 1 -- 172.21.15.40:0/99038292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3344141320 msgr2=0x7f334413e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:20.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.954+0000 7f334a23a640 1 --2- 172.21.15.40:0/99038292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3344141320 0x7f334413e1d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3344066830 tx=0x7f333c02bee0 comp rx=0 tx=0).stop 2023-07-29T13:28:20.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.954+0000 7f334a23a640 1 -- 172.21.15.40:0/99038292 shutdown_connections 2023-07-29T13:28:20.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.954+0000 7f334a23a640 1 --2- 172.21.15.40:0/99038292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3344141320 0x7f334413e1d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:20.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.954+0000 7f334a23a640 1 --2- 172.21.15.40:0/99038292 >> v2:172.21.15.45:3300/0 conn(0x7f33440b3ae0 0x7f334413dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:20.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.954+0000 7f334a23a640 1 -- 172.21.15.40:0/99038292 >> 172.21.15.40:0/99038292 conn(0x7f3344070fc0 msgr2=0x7f33440713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:20.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.954+0000 7f334a23a640 1 -- 172.21.15.40:0/99038292 shutdown_connections 2023-07-29T13:28:20.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.954+0000 7f334a23a640 1 -- 172.21.15.40:0/99038292 wait complete. 2023-07-29T13:28:20.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.955+0000 7f334a23a640 10 monclient: build_initial_monmap 2023-07-29T13:28:20.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.955+0000 7f334a23a640 10 monclient: monmap: 2023-07-29T13:28:20.963 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:20.963 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:20.963 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:20.963 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:20.963 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:20.963 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:20.963 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:20.964 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:20.964 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:20.964 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:20.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.955+0000 7f334a23a640 1 librados: starting msgr at 2023-07-29T13:28:20.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.955+0000 7f334a23a640 1 librados: starting objecter 2023-07-29T13:28:20.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.955+0000 7f334a23a640 1 Processor -- start 2023-07-29T13:28:20.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 1 -- start start 2023-07-29T13:28:20.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 1 librados: setting wanted keys 2023-07-29T13:28:20.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 1 librados: calling monclient init 2023-07-29T13:28:20.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 10 monclient: init 2023-07-29T13:28:20.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:20.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T13:28:20.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440a8910 0x7f33441d6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:20.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 10 monclient(hunting): picked mon.noname-a con 0x7f33440a8910 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:20.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33440b3ae0 0x7f33441443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 10 monclient(hunting): picked mon.noname-b con 0x7f33440b3ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3344141320 0x7f33441d57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 10 monclient(hunting): picked mon.noname-c con 0x7f3344141320 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33440ad530 con 0x7f33440a8910 2023-07-29T13:28:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33441d5e30 con 0x7f33440b3ae0 2023-07-29T13:28:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33440b6940 con 0x7f3344141320 2023-07-29T13:28:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.956+0000 7f334a23a640 10 monclient(hunting): authenticate will time out at 1423.089887s 2023-07-29T13:28:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.958+0000 7f33437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440a8910 0x7f33441d6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.958+0000 7f33437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440a8910 0x7f33441d6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33118/0 (socket says 172.21.15.40:33118) 2023-07-29T13:28:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.958+0000 7f33437fe640 1 -- 172.21.15.40:0/4252559993 learned_addr learned my addr 172.21.15.40:0/4252559993 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.958+0000 7f33437fe640 10 monclient(hunting): get_auth_request con 0x7f33440a8910 auth_method 0 2023-07-29T13:28:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.958+0000 7f33437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:20.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.958+0000 7f33437fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:20.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.958+0000 7f33437fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.960+0000 7f3342ffd640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33440b3ae0 0x7f33441443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.960+0000 7f3342ffd640 10 monclient(hunting): get_auth_request con 0x7f33440b3ae0 auth_method 0 2023-07-29T13:28:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.960+0000 7f3342ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.960+0000 7f3342ffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.960+0000 7f3342ffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.961+0000 7f3342ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.961+0000 7f3342ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:20.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.961+0000 7f3342ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:20.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.966+0000 7f3343fff640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3344141320 0x7f33441d57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:20.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.967+0000 7f3343fff640 10 monclient(hunting): get_auth_request con 0x7f3344141320 auth_method 0 2023-07-29T13:28:20.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.967+0000 7f3343fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:20.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.967+0000 7f3343fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:20.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.967+0000 7f3343fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:20.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.968+0000 7f3343fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:20.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.968+0000 7f3343fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:20.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.968+0000 7f3343fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:20.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.971+0000 7f3343fff640 10 monclient(hunting): handle_auth_done global_id 4283 payload 824 2023-07-29T13:28:20.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.971+0000 7f3343fff640 1 -- 172.21.15.40:0/4252559993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33440b3ae0 msgr2=0x7f33441443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:20.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.971+0000 7f3343fff640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33440b3ae0 0x7f33441443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:20.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.971+0000 7f3343fff640 1 -- 172.21.15.40:0/4252559993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440a8910 msgr2=0x7f33441d6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:20.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.971+0000 7f3343fff640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440a8910 0x7f33441d6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:20.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.971+0000 7f3343fff640 10 monclient: _finish_hunting 0 2023-07-29T13:28:20.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.971+0000 7f3343fff640 1 monclient: found mon.noname-c 2023-07-29T13:28:20.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.971+0000 7f3343fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:20.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.971+0000 7f3343fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T13:28:20.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.971+0000 7f3343fff640 1 -- 172.21.15.40:0/4252559993 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c009b30 con 0x7f3344141320 2023-07-29T13:28:20.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.974+0000 7f3343fff640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3344141320 0x7f33441d57d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f333c032000 tx=0x7f333c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:20.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.974+0000 7f332bfff640 10 client.?.objecter ms_handle_connect 0x7f3344141320 2023-07-29T13:28:20.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:20.974+0000 7f332bfff640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 1 -- 172.21.15.40:0/4252559993 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c03e450 con 0x7f3344141320 2023-07-29T13:28:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 10 monclient: dump: 2023-07-29T13:28:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:21.043 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 10 monclient: _finish_auth 0 2023-07-29T13:28:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:28:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 1 -- 172.21.15.40:0/4252559993 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c03e5b0 con 0x7f3344141320 2023-07-29T13:28:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 1 -- 172.21.15.40:0/4252559993 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c03e9a0 con 0x7f3344141320 2023-07-29T13:28:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f334a23a640 5 monclient: authenticate success, global_id 4283 2023-07-29T13:28:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f334a23a640 10 monclient: _renew_subs 2023-07-29T13:28:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f334a23a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f334a23a640 1 -- 172.21.15.40:0/4252559993 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33440aa050 con 0x7f3344141320 2023-07-29T13:28:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f332bfff640 10 monclient: dump: 2023-07-29T13:28:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.046 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:21.046 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:21.046 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:21.046 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:21.046 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:21.046 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:21.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f3341ffb640 4 set_mon_vals no callback set 2023-07-29T13:28:21.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f334a23a640 10 client.4283.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:21.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f334a23a640 10 monclient: _renew_subs 2023-07-29T13:28:21.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f334a23a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:21.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f334a23a640 1 -- 172.21.15.40:0/4252559993 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33440aa530 con 0x7f3344141320 2023-07-29T13:28:21.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.039+0000 7f334a23a640 1 librados: init done 2023-07-29T13:28:21.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.040+0000 7f334a23a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:21.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.040+0000 7f334a23a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:21.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.040+0000 7f334a23a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:21.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.040+0000 7f334a23a640 1 -- 172.21.15.40:0/4252559993 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33440ad670 con 0x7f3344141320 2023-07-29T13:28:21.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.347+0000 7f332bfff640 1 -- 172.21.15.40:0/4252559993 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f333c02cd50 con 0x7f3344141320 2023-07-29T13:28:21.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.348+0000 7f332bfff640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f333006bf60 0x7f333006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:21.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.348+0000 7f33437fe640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f333006bf60 0x7f333006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:21.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.348+0000 7f33437fe640 10 monclient: get_auth_request con 0x7f333006bf60 auth_method 0 2023-07-29T13:28:21.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.349+0000 7f332bfff640 10 client.4283.objecter ms_handle_connect 0x7f333006bf60 2023-07-29T13:28:21.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.349+0000 7f33437fe640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f333006bf60 0x7f333006e410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3334009b90 tx=0x7f3334008040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.365+0000 7f332bfff640 1 -- 172.21.15.40:0/4252559993 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f333c07e720 con 0x7f3344141320 2023-07-29T13:28:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.365+0000 7f332bfff640 10 client.4283.objecter ms_dispatch 0x7f3344000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.365+0000 7f332bfff640 3 client.4283.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.365+0000 7f332bfff640 3 client.4283.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.365+0000 7f332bfff640 20 client.4283.objecter dump_active .. 0 homeless 2023-07-29T13:28:21.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.365+0000 7f332bfff640 1 -- 172.21.15.40:0/4252559993 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f333c076aa0 con 0x7f3344141320 2023-07-29T13:28:21.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.365+0000 7f332bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:21.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.365+0000 7f332bfff640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.548+0000 7f334a23a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2023-07-29T13:28:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.548+0000 7f334a23a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-07-29T13:28:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.548+0000 7f334a23a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T13:28:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.548+0000 7f334a23a640 1 -- 172.21.15.40:0/4252559993 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f33440ad670 con 0x7f3344141320 2023-07-29T13:28:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.552+0000 7f332bfff640 1 -- 172.21.15.40:0/4252559993 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f333c043020 con 0x7f3344141320 2023-07-29T13:28:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.552+0000 7f332bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-07-29T13:28:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.552+0000 7f332bfff640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:21.554 INFO:teuthology.orchestra.run.smithi040.stdout:68719476740 2023-07-29T13:28:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 10 librados: watch_flush enter 2023-07-29T13:28:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 10 librados: watch_flush exit 2023-07-29T13:28:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 20 client.4283.objecter shutdown clearing up homeless session... 2023-07-29T13:28:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 10 client.4283.objecter successfully canceled tick 2023-07-29T13:28:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 -- 172.21.15.40:0/4252559993 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f333006bf60 msgr2=0x7f333006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f333006bf60 0x7f333006e410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3334009b90 tx=0x7f3334008040 comp rx=0 tx=0).stop 2023-07-29T13:28:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 10 monclient: shutdown 2023-07-29T13:28:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 -- 172.21.15.40:0/4252559993 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3344141320 msgr2=0x7f33441d57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3344141320 0x7f33441d57d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f333c032000 tx=0x7f333c004a40 comp rx=0 tx=0).stop 2023-07-29T13:28:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 -- 172.21.15.40:0/4252559993 shutdown_connections 2023-07-29T13:28:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f333006bf60 0x7f333006e410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33440b3ae0 0x7f33441443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3344141320 0x7f33441d57d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 --2- 172.21.15.40:0/4252559993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440a8910 0x7f33441d6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 -- 172.21.15.40:0/4252559993 >> 172.21.15.40:0/4252559993 conn(0x7f3344070fc0 msgr2=0x7f33440b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 -- 172.21.15.40:0/4252559993 shutdown_connections 2023-07-29T13:28:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 -- 172.21.15.40:0/4252559993 wait complete. 2023-07-29T13:28:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.554+0000 7f334a23a640 1 librados: shutdown 2023-07-29T13:28:21.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476740 for osd.4 2023-07-29T13:28:21.567 DEBUG:teuthology.parallel:result is None 2023-07-29T13:28:21.567 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-29T13:28:21.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-29T13:28:21.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.673+0000 7f1321426640 10 monclient: get_monmap_and_config 2023-07-29T13:28:21.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.673+0000 7f1321426640 10 monclient: build_initial_monmap 2023-07-29T13:28:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.673+0000 7f1321426640 10 monclient: monmap: 2023-07-29T13:28:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:21.675068+0000 2023-07-29T13:28:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:21.675068+0000 2023-07-29T13:28:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:21.677 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:21.677 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:21.677 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:21.677 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:21.677 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:21.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.674+0000 7f1321426640 1 Processor -- start 2023-07-29T13:28:21.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.674+0000 7f1321426640 1 -- start start 2023-07-29T13:28:21.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.674+0000 7f1321426640 10 monclient: init 2023-07-29T13:28:21.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:21.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2023-07-29T13:28:21.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c0b74a0 0x7f131c0aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:21.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 10 monclient(hunting): picked mon.noname-b con 0x7f131c0b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:21.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f131c13dde0 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:21.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f131c1445e0 0x7f131c144a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:21.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 10 monclient(hunting): picked mon.noname-c con 0x7f131c1445e0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:21.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131c0b7870 con 0x7f131c13dde0 2023-07-29T13:28:21.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:21.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131c0a91b0 con 0x7f131c1445e0 2023-07-29T13:28:21.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:21.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0a9490 con 0x7f131c0b74a0 2023-07-29T13:28:21.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:21.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f1321426640 10 monclient(hunting): authenticate will time out at 1423.808221s 2023-07-29T13:28:21.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f131affd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c0b74a0 0x7f131c0aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:21.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.675+0000 7f131b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f131c1445e0 0x7f131c144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:21.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131affd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c0b74a0 0x7f131c0aa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:47258/0 (socket says 172.21.15.40:47258) 2023-07-29T13:28:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131affd640 1 -- 172.21.15.40:0/1880430791 learned_addr learned my addr 172.21.15.40:0/1880430791 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131affd640 10 monclient(hunting): get_auth_request con 0x7f131c0b74a0 auth_method 0 2023-07-29T13:28:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131affd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131affd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131b7fe640 10 monclient(hunting): get_auth_request con 0x7f131c1445e0 auth_method 0 2023-07-29T13:28:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131b7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.676+0000 7f131affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.677+0000 7f131b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.677+0000 7f131b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.677+0000 7f131b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.678+0000 7f131affd640 10 monclient(hunting): handle_auth_done global_id 4286 payload 274 2023-07-29T13:28:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.678+0000 7f131affd640 1 -- 172.21.15.40:0/1880430791 >> v2:172.21.15.45:3300/0 conn(0x7f131c1445e0 msgr2=0x7f131c144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.678+0000 7f131affd640 1 --2- 172.21.15.40:0/1880430791 >> v2:172.21.15.45:3300/0 conn(0x7f131c1445e0 0x7f131c144a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.678+0000 7f131affd640 1 -- 172.21.15.40:0/1880430791 >> v1:172.21.15.45:6789/0 conn(0x7f131c13dde0 legacy=0x7f131c13e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.678+0000 7f131affd640 10 monclient: _finish_hunting 0 2023-07-29T13:28:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.678+0000 7f131affd640 1 monclient: found mon.noname-b 2023-07-29T13:28:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.678+0000 7f131affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.678+0000 7f131affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T13:28:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.678+0000 7f131affd640 1 -- 172.21.15.40:0/1880430791 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c0a9770 con 0x7f131c0b74a0 2023-07-29T13:28:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.680+0000 7f131affd640 1 --2- 172.21.15.40:0/1880430791 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c0b74a0 0x7f131c0aa150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f130c009850 tx=0x7f130c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad9f3838aa081f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 1 -- 172.21.15.40:0/1880430791 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c004230 con 0x7f131c0b74a0 2023-07-29T13:28:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 10 monclient: dump: 2023-07-29T13:28:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:21.687 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 10 monclient: _finish_auth 0 2023-07-29T13:28:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 10 monclient: _check_auth_tickets 2023-07-29T13:28:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 1 -- 172.21.15.40:0/1880430791 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130c004390 con 0x7f131c0b74a0 2023-07-29T13:28:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 1 -- 172.21.15.40:0/1880430791 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c031830 con 0x7f131c0b74a0 2023-07-29T13:28:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T13:28:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f131a7fc640 10 monclient: dump: 2023-07-29T13:28:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:21.690 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f1321426640 5 monclient: authenticate success, global_id 4286 2023-07-29T13:28:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f1321426640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.684+0000 7f1321426640 4 set_mon_vals no callback set 2023-07-29T13:28:21.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 10 monclient: shutdown 2023-07-29T13:28:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 1 -- 172.21.15.40:0/1880430791 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c0b74a0 msgr2=0x7f131c0aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 1 --2- 172.21.15.40:0/1880430791 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c0b74a0 0x7f131c0aa150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f130c009850 tx=0x7f130c02b8d0 comp rx=0 tx=0).stop 2023-07-29T13:28:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 1 -- 172.21.15.40:0/1880430791 shutdown_connections 2023-07-29T13:28:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 1 --2- 172.21.15.40:0/1880430791 >> v2:172.21.15.45:3300/0 conn(0x7f131c1445e0 0x7f131c144a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 1 --2- 172.21.15.40:0/1880430791 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c0b74a0 0x7f131c0aa150 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 1 -- 172.21.15.40:0/1880430791 >> 172.21.15.40:0/1880430791 conn(0x7f131c065b00 msgr2=0x7f131c065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:21.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 1 -- 172.21.15.40:0/1880430791 shutdown_connections 2023-07-29T13:28:21.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 1 -- 172.21.15.40:0/1880430791 wait complete. 2023-07-29T13:28:21.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 10 monclient: build_initial_monmap 2023-07-29T13:28:21.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.685+0000 7f1321426640 10 monclient: monmap: 2023-07-29T13:28:21.694 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:21.694 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:21.694 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:21.694 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:21.695 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:21.695 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:21.695 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:21.695 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:21.695 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:21.695 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:21.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.686+0000 7f1321426640 1 librados: starting msgr at 2023-07-29T13:28:21.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.686+0000 7f1321426640 1 librados: starting objecter 2023-07-29T13:28:21.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.686+0000 7f1321426640 1 Processor -- start 2023-07-29T13:28:21.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.686+0000 7f1321426640 1 -- start start 2023-07-29T13:28:21.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.686+0000 7f1321426640 1 librados: setting wanted keys 2023-07-29T13:28:21.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.686+0000 7f1321426640 1 librados: calling monclient init 2023-07-29T13:28:21.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.686+0000 7f1321426640 10 monclient: init 2023-07-29T13:28:21.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:21.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T13:28:21.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f131c0b74a0 0x7f131c0b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:21.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 10 monclient(hunting): picked mon.noname-b con 0x7f131c0b74a0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:21.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c13dde0 0x7f131c133f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:21.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 10 monclient(hunting): picked mon.noname-c con 0x7f131c13dde0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:21.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c1445e0 0x7f131c1344c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:21.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 10 monclient(hunting): picked mon.noname-a con 0x7f131c1445e0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:21.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:21.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c1439a0 con 0x7f131c1445e0 2023-07-29T13:28:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c144110 con 0x7f131c0b74a0 2023-07-29T13:28:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c143bf0 con 0x7f131c13dde0 2023-07-29T13:28:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f1321426640 10 monclient(hunting): authenticate will time out at 1423.820301s 2023-07-29T13:28:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f13127fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c13dde0 0x7f131c133f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f131b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c1445e0 0x7f131c1344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.687+0000 7f131affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f131c0b74a0 0x7f131c0b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f13127fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c13dde0 0x7f131c133f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:47264/0 (socket says 172.21.15.40:47264) 2023-07-29T13:28:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f13127fc640 1 -- 172.21.15.40:0/783694855 learned_addr learned my addr 172.21.15.40:0/783694855 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f13127fc640 10 monclient(hunting): get_auth_request con 0x7f131c13dde0 auth_method 0 2023-07-29T13:28:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f13127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f13127fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f13127fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131b7fe640 10 monclient(hunting): get_auth_request con 0x7f131c1445e0 auth_method 0 2023-07-29T13:28:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131b7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131affd640 10 monclient(hunting): get_auth_request con 0x7f131c0b74a0 auth_method 0 2023-07-29T13:28:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131affd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131affd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f13127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:21.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f13127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:21.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f13127fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:21.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:21.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:21.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:21.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:21.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:21.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.688+0000 7f131b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:21.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.691+0000 7f131affd640 10 monclient(hunting): handle_auth_done global_id 4345 payload 824 2023-07-29T13:28:21.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.691+0000 7f131affd640 1 -- 172.21.15.40:0/783694855 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c13dde0 msgr2=0x7f131c133f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:21.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.691+0000 7f131affd640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c13dde0 0x7f131c133f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:21.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.691+0000 7f131affd640 1 -- 172.21.15.40:0/783694855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c1445e0 msgr2=0x7f131c1344c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.691+0000 7f131affd640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c1445e0 0x7f131c1344c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.691+0000 7f131affd640 10 monclient: _finish_hunting 0 2023-07-29T13:28:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.691+0000 7f131affd640 1 monclient: found mon.noname-b 2023-07-29T13:28:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.691+0000 7f131affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.691+0000 7f131affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.691+0000 7f131affd640 1 -- 172.21.15.40:0/783694855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c009460 con 0x7f131c0b74a0 2023-07-29T13:28:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.693+0000 7f131affd640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f131c0b74a0 0x7f131c0b38b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f130c004000 tx=0x7f130c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:21.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.693+0000 7f1313fff640 10 client.?.objecter ms_handle_connect 0x7f131c0b74a0 2023-07-29T13:28:21.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.693+0000 7f1313fff640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:21.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 1 -- 172.21.15.40:0/783694855 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c031630 con 0x7f131c0b74a0 2023-07-29T13:28:21.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:21.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:21.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 10 monclient: dump: 2023-07-29T13:28:21.705 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:21.705 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:21.706 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.706 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.706 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:21.706 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:21.706 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:21.706 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:21.706 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:21.706 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 10 monclient: _finish_auth 0 2023-07-29T13:28:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 10 monclient: _check_auth_tickets 2023-07-29T13:28:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 1 -- 172.21.15.40:0/783694855 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130c03f070 con 0x7f131c0b74a0 2023-07-29T13:28:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 1 -- 172.21.15.40:0/783694855 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c0319f0 con 0x7f131c0b74a0 2023-07-29T13:28:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:21.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1313fff640 10 monclient: dump: 2023-07-29T13:28:21.708 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:21.708 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:21.708 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.708 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:21.708 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:21.708 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:21.709 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1321426640 5 monclient: authenticate success, global_id 4345 2023-07-29T13:28:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1321426640 10 monclient: _renew_subs 2023-07-29T13:28:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1319ffb640 4 set_mon_vals no callback set 2023-07-29T13:28:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1321426640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.697+0000 7f1321426640 1 -- 172.21.15.40:0/783694855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c0b4350 con 0x7f131c0b74a0 2023-07-29T13:28:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.698+0000 7f1321426640 10 client.4345.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.698+0000 7f1321426640 10 monclient: _renew_subs 2023-07-29T13:28:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.698+0000 7f1321426640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.698+0000 7f1321426640 1 -- 172.21.15.40:0/783694855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131c138df0 con 0x7f131c0b74a0 2023-07-29T13:28:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.698+0000 7f1321426640 1 librados: init done 2023-07-29T13:28:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.698+0000 7f1321426640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.698+0000 7f1321426640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:21.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.698+0000 7f1321426640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:21.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.698+0000 7f1321426640 1 -- 172.21.15.40:0/783694855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131c04e000 con 0x7f131c0b74a0 2023-07-29T13:28:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.718+0000 7f1313fff640 1 -- 172.21.15.40:0/783694855 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f130c041030 con 0x7f131c0b74a0 2023-07-29T13:28:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.718+0000 7f1313fff640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f12fc06bfb0 0x7f12fc06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.719+0000 7f13127fc640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f12fc06bfb0 0x7f12fc06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.719+0000 7f13127fc640 10 monclient: get_auth_request con 0x7f12fc06bfb0 auth_method 0 2023-07-29T13:28:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.719+0000 7f13127fc640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f12fc06bfb0 0x7f12fc06e460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f131c13b4f0 tx=0x7f1304009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.719+0000 7f1313fff640 10 client.4345.objecter ms_handle_connect 0x7f12fc06bfb0 2023-07-29T13:28:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.720+0000 7f1313fff640 1 -- 172.21.15.40:0/783694855 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f130c07e320 con 0x7f131c0b74a0 2023-07-29T13:28:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.720+0000 7f1313fff640 10 client.4345.objecter ms_dispatch 0x7f131c000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.720+0000 7f1313fff640 3 client.4345.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.720+0000 7f1313fff640 3 client.4345.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.720+0000 7f1313fff640 20 client.4345.objecter dump_active .. 0 homeless 2023-07-29T13:28:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.966+0000 7f1313fff640 1 -- 172.21.15.40:0/783694855 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f130c076750 con 0x7f131c0b74a0 2023-07-29T13:28:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.966+0000 7f1313fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:21.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:21.966+0000 7f1313fff640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:22.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.142+0000 7f1321426640 1 -- 172.21.15.40:0/783694855 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f131c138fe0 con 0x7f12fc06bfb0 2023-07-29T13:28:22.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.144+0000 7f1313fff640 1 -- 172.21.15.40:0/783694855 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47959 (secure 0 0 0) 0x7f131c138fe0 con 0x7f12fc06bfb0 2023-07-29T13:28:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:28:22.147 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-29T13:28:22.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 10 librados: watch_flush enter 2023-07-29T13:28:22.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 10 librados: watch_flush exit 2023-07-29T13:28:22.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 20 client.4345.objecter shutdown clearing up homeless session... 2023-07-29T13:28:22.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 10 client.4345.objecter successfully canceled tick 2023-07-29T13:28:22.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 -- 172.21.15.40:0/783694855 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f12fc06bfb0 msgr2=0x7f12fc06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f12fc06bfb0 0x7f12fc06e460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f131c13b4f0 tx=0x7f1304009040 comp rx=0 tx=0).stop 2023-07-29T13:28:22.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 10 monclient: shutdown 2023-07-29T13:28:22.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:22.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 -- 172.21.15.40:0/783694855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f131c0b74a0 msgr2=0x7f131c0b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f131c0b74a0 0x7f131c0b38b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f130c004000 tx=0x7f130c0046c0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 -- 172.21.15.40:0/783694855 shutdown_connections 2023-07-29T13:28:22.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f12fc06bfb0 0x7f12fc06e460 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c1445e0 0x7f131c1344c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f131c13dde0 0x7f131c133f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 --2- 172.21.15.40:0/783694855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f131c0b74a0 0x7f131c0b38b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 -- 172.21.15.40:0/783694855 >> 172.21.15.40:0/783694855 conn(0x7f131c065b00 msgr2=0x7f131c13d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:22.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 -- 172.21.15.40:0/783694855 shutdown_connections 2023-07-29T13:28:22.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 -- 172.21.15.40:0/783694855 wait complete. 2023-07-29T13:28:22.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.146+0000 7f1321426640 1 librados: shutdown 2023-07-29T13:28:22.158 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-07-29T13:28:21.016427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218256,"kb_used_data":2640,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756464,"statfs":{"total":773094113280,"available":772870619136,"internally_reserved":0,"allocated":2703360,"data_stored":1441616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.137619"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.038384+0000","last_change":"2023-07-29T13:28:14.694035+0000","last_active":"2023-07-29T13:28:15.038384+0000","last_peered":"2023-07-29T13:28:15.038384+0000","last_clean":"2023-07-29T13:28:15.038384+0000","last_became_active":"2023-07-29T13:28:14.688540+0000","last_became_peered":"2023-07-29T13:28:14.688540+0000","last_unstale":"2023-07-29T13:28:15.038384+0000","last_undegraded":"2023-07-29T13:28:15.038384+0000","last_fullsized":"2023-07-29T13:28:15.038384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:31:19.269462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.828590+0000","last_change":"2023-07-29T13:28:15.118143+0000","last_active":"2023-07-29T13:28:15.828590+0000","last_peered":"2023-07-29T13:28:15.828590+0000","last_clean":"2023-07-29T13:28:15.828590+0000","last_became_active":"2023-07-29T13:28:15.089605+0000","last_became_peered":"2023-07-29T13:28:15.089605+0000","last_unstale":"2023-07-29T13:28:15.828590+0000","last_undegraded":"2023-07-29T13:28:15.828590+0000","last_fullsized":"2023-07-29T13:28:15.828590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:00:18.108603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.047826+0000","last_change":"2023-07-29T13:28:14.434312+0000","last_active":"2023-07-29T13:28:15.047826+0000","last_peered":"2023-07-29T13:28:15.047826+0000","last_clean":"2023-07-29T13:28:15.047826+0000","last_became_active":"2023-07-29T13:28:14.423044+0000","last_became_peered":"2023-07-29T13:28:14.423044+0000","last_unstale":"2023-07-29T13:28:15.047826+0000","last_undegraded":"2023-07-29T13:28:15.047826+0000","last_fullsized":"2023-07-29T13:28:15.047826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:56:27.865715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.042026+0000","last_change":"2023-07-29T13:28:14.686642+0000","last_active":"2023-07-29T13:28:15.042026+0000","last_peered":"2023-07-29T13:28:15.042026+0000","last_clean":"2023-07-29T13:28:15.042026+0000","last_became_active":"2023-07-29T13:28:14.659427+0000","last_became_peered":"2023-07-29T13:28:14.659427+0000","last_unstale":"2023-07-29T13:28:15.042026+0000","last_undegraded":"2023-07-29T13:28:15.042026+0000","last_fullsized":"2023-07-29T13:28:15.042026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:24:15.162936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.029326+0000","last_change":"2023-07-29T13:28:14.652586+0000","last_active":"2023-07-29T13:28:15.029326+0000","last_peered":"2023-07-29T13:28:15.029326+0000","last_clean":"2023-07-29T13:28:15.029326+0000","last_became_active":"2023-07-29T13:28:14.642023+0000","last_became_peered":"2023-07-29T13:28:14.642023+0000","last_unstale":"2023-07-29T13:28:15.029326+0000","last_undegraded":"2023-07-29T13:28:15.029326+0000","last_fullsized":"2023-07-29T13:28:15.029326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:25:23.334956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:14.889983+0000","last_change":"2023-07-29T13:28:14.256724+0000","last_active":"2023-07-29T13:28:14.889983+0000","last_peered":"2023-07-29T13:28:14.889983+0000","last_clean":"2023-07-29T13:28:14.889983+0000","last_became_active":"2023-07-29T13:28:14.228272+0000","last_became_peered":"2023-07-29T13:28:14.228272+0000","last_unstale":"2023-07-29T13:28:14.889983+0000","last_undegraded":"2023-07-29T13:28:14.889983+0000","last_fullsized":"2023-07-29T13:28:14.889983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:24:42.037027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.041783+0000","last_change":"2023-07-29T13:28:14.384771+0000","last_active":"2023-07-29T13:28:15.041783+0000","last_peered":"2023-07-29T13:28:15.041783+0000","last_clean":"2023-07-29T13:28:15.041783+0000","last_became_active":"2023-07-29T13:28:14.353364+0000","last_became_peered":"2023-07-29T13:28:14.353364+0000","last_unstale":"2023-07-29T13:28:15.041783+0000","last_undegraded":"2023-07-29T13:28:15.041783+0000","last_fullsized":"2023-07-29T13:28:15.041783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:28:57.053189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.034836+0000","last_change":"2023-07-29T13:28:14.713485+0000","last_active":"2023-07-29T13:28:15.034836+0000","last_peered":"2023-07-29T13:28:15.034836+0000","last_clean":"2023-07-29T13:28:15.034836+0000","last_became_active":"2023-07-29T13:28:14.708042+0000","last_became_peered":"2023-07-29T13:28:14.708042+0000","last_unstale":"2023-07-29T13:28:15.034836+0000","last_undegraded":"2023-07-29T13:28:15.034836+0000","last_fullsized":"2023-07-29T13:28:15.034836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:25:56.525512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"21'64","reported_seq":103,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.003696+0000","last_change":"2023-07-29T13:28:10.113325+0000","last_active":"2023-07-29T13:28:15.003696+0000","last_peered":"2023-07-29T13:28:15.003696+0000","last_clean":"2023-07-29T13:28:15.003696+0000","last_became_active":"2023-07-29T13:28:10.081018+0000","last_became_peered":"2023-07-29T13:28:10.081018+0000","last_unstale":"2023-07-29T13:28:15.003696+0000","last_undegraded":"2023-07-29T13:28:15.003696+0000","last_fullsized":"2023-07-29T13:28:15.003696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_clean_scrub_stamp":"2023-07-29T13:28:06.434639+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:30:49.511284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":17,"seq":73014444038,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.481}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.553000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.323}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.346}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.866}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.579000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":25.216999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.2720000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.7509999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.811999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.3889999999999993}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.686999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.103999999999999}]}]},{"osd":1,"up_from":17,"seq":73014444039,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":478582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.928000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.404}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":31.324999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.356}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":37.991999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.266}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.603000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444039,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.337}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.655999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.238}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":27.369}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.73}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.853000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.436}]}]},{"osd":4,"up_from":16,"seq":68719476741,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.238}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.239000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.9130000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.244}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.343999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.911999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.378}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4019999999999992}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.190000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.834}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.776}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.959}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.588000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":478582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5969999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.105}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.7940000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.31}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.083}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.664}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.018999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.3689999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.223000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.343999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.756}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.170999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.978999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-29T13:28:22.160 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-29T13:28:22.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.263+0000 7f7d381b9640 10 monclient: get_monmap_and_config 2023-07-29T13:28:22.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.263+0000 7f7d381b9640 10 monclient: build_initial_monmap 2023-07-29T13:28:22.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.263+0000 7f7d381b9640 10 monclient: monmap: 2023-07-29T13:28:22.266 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:22.266 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:22.266 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:22.265072+0000 2023-07-29T13:28:22.266 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:22.265072+0000 2023-07-29T13:28:22.266 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:22.266 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.266 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:22.267 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:22.267 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:22.267 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:22.267 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:22.267 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.264+0000 7f7d381b9640 1 Processor -- start 2023-07-29T13:28:22.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.264+0000 7f7d381b9640 1 -- start start 2023-07-29T13:28:22.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.264+0000 7f7d381b9640 10 monclient: init 2023-07-29T13:28:22.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:22.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2023-07-29T13:28:22.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7d300b74a0 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:22.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d3013e4f0 0x7f7d300aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:22.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 10 monclient(hunting): picked mon.noname-c con 0x7f7d3013e4f0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:22.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d301445e0 0x7f7d30144a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:22.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 10 monclient(hunting): picked mon.noname-a con 0x7f7d301445e0 addr v2:172.21.15.40:3300/0 2023-07-29T13:28:22.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d300b7870 con 0x7f7d300b74a0 2023-07-29T13:28:22.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:22.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d300a91b0 con 0x7f7d301445e0 2023-07-29T13:28:22.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:22.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d300a9490 con 0x7f7d3013e4f0 2023-07-29T13:28:22.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:22.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.265+0000 7f7d381b9640 10 monclient(hunting): authenticate will time out at 1424.398489s 2023-07-29T13:28:22.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.266+0000 7f7d36756640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d301445e0 0x7f7d30144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:22.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.266+0000 7f7d35754640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d3013e4f0 0x7f7d300aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:22.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.266+0000 7f7d36756640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d301445e0 0x7f7d30144a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33126/0 (socket says 172.21.15.40:33126) 2023-07-29T13:28:22.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.266+0000 7f7d36756640 1 -- 172.21.15.40:0/1314313417 learned_addr learned my addr 172.21.15.40:0/1314313417 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:22.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.266+0000 7f7d36756640 10 monclient(hunting): get_auth_request con 0x7f7d301445e0 auth_method 0 2023-07-29T13:28:22.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.266+0000 7f7d36756640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:22.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.266+0000 7f7d36756640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:22.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.266+0000 7f7d36756640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:22.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.266+0000 7f7d36756640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:22.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.266+0000 7f7d36756640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:22.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.266+0000 7f7d36756640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:22.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.267+0000 7f7d35754640 10 monclient(hunting): get_auth_request con 0x7f7d3013e4f0 auth_method 0 2023-07-29T13:28:22.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.267+0000 7f7d35754640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:22.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.267+0000 7f7d35754640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:22.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.267+0000 7f7d35754640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:22.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.267+0000 7f7d35754640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:22.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.267+0000 7f7d35754640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:22.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.267+0000 7f7d35754640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:22.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.268+0000 7f7d36756640 10 monclient(hunting): handle_auth_done global_id 4377 payload 274 2023-07-29T13:28:22.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.268+0000 7f7d36756640 1 -- 172.21.15.40:0/1314313417 >> v2:172.21.15.45:3300/0 conn(0x7f7d3013e4f0 msgr2=0x7f7d300aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:22.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.268+0000 7f7d36756640 1 --2- 172.21.15.40:0/1314313417 >> v2:172.21.15.45:3300/0 conn(0x7f7d3013e4f0 0x7f7d300aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.268+0000 7f7d36756640 1 -- 172.21.15.40:0/1314313417 >> v1:172.21.15.45:6789/0 conn(0x7f7d300b74a0 legacy=0x7f7d3013dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.268+0000 7f7d36756640 10 monclient: _finish_hunting 0 2023-07-29T13:28:22.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.268+0000 7f7d36756640 1 monclient: found mon.noname-a 2023-07-29T13:28:22.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.268+0000 7f7d36756640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:22.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.268+0000 7f7d36756640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:28:22.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.268+0000 7f7d36756640 1 -- 172.21.15.40:0/1314313417 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d300a9770 con 0x7f7d301445e0 2023-07-29T13:28:22.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.270+0000 7f7d36756640 1 --2- 172.21.15.40:0/1314313417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d301445e0 0x7f7d30144a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d300b7cf0 tx=0x7f7d2002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7c94d3d02e51dd server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:22.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 1 -- 172.21.15.40:0/1314313417 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2002ca80 con 0x7f7d301445e0 2023-07-29T13:28:22.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:22.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:28:22.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 10 monclient: dump: 2023-07-29T13:28:22.277 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:22.277 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:22.277 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.278 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.278 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:22.278 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.278 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:22.278 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:22.278 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:22.278 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 10 monclient: _finish_auth 0 2023-07-29T13:28:22.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 10 monclient: _check_auth_tickets 2023-07-29T13:28:22.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:22.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 1 -- 172.21.15.40:0/1314313417 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2002cbe0 con 0x7f7d301445e0 2023-07-29T13:28:22.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d381b9640 5 monclient: authenticate success, global_id 4377 2023-07-29T13:28:22.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d381b9640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:28:22.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:22.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d381b9640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:22.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 1 -- 172.21.15.40:0/1314313417 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d200343f0 con 0x7f7d301445e0 2023-07-29T13:28:22.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d381b9640 4 set_mon_vals no callback set 2023-07-29T13:28:22.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:22.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:28:22.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.274+0000 7f7d34f53640 10 monclient: dump: 2023-07-29T13:28:22.280 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:22.280 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:22.280 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.281 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.281 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:22.281 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.281 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:22.281 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:22.281 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:22.281 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.275+0000 7f7d381b9640 10 monclient: shutdown 2023-07-29T13:28:22.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.275+0000 7f7d381b9640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:22.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.275+0000 7f7d381b9640 1 -- 172.21.15.40:0/1314313417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d301445e0 msgr2=0x7f7d30144a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.275+0000 7f7d381b9640 1 --2- 172.21.15.40:0/1314313417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d301445e0 0x7f7d30144a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d300b7cf0 tx=0x7f7d2002bee0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.275+0000 7f7d381b9640 1 -- 172.21.15.40:0/1314313417 shutdown_connections 2023-07-29T13:28:22.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.275+0000 7f7d381b9640 1 --2- 172.21.15.40:0/1314313417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d301445e0 0x7f7d30144a60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.275+0000 7f7d381b9640 1 --2- 172.21.15.40:0/1314313417 >> v2:172.21.15.45:3300/0 conn(0x7f7d3013e4f0 0x7f7d300aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.275+0000 7f7d381b9640 1 -- 172.21.15.40:0/1314313417 >> 172.21.15.40:0/1314313417 conn(0x7f7d30065b00 msgr2=0x7f7d30065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:22.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.275+0000 7f7d381b9640 1 -- 172.21.15.40:0/1314313417 shutdown_connections 2023-07-29T13:28:22.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.275+0000 7f7d381b9640 1 -- 172.21.15.40:0/1314313417 wait complete. 2023-07-29T13:28:22.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.276+0000 7f7d381b9640 10 monclient: build_initial_monmap 2023-07-29T13:28:22.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.276+0000 7f7d381b9640 10 monclient: monmap: 2023-07-29T13:28:22.287 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:22.288 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:22.288 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:22.288 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:22.289 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:22.289 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.289 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:22.289 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:22.289 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:22.289 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.276+0000 7f7d381b9640 1 librados: starting msgr at 2023-07-29T13:28:22.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.276+0000 7f7d381b9640 1 librados: starting objecter 2023-07-29T13:28:22.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.276+0000 7f7d381b9640 1 Processor -- start 2023-07-29T13:28:22.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.276+0000 7f7d381b9640 1 -- start start 2023-07-29T13:28:22.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.276+0000 7f7d381b9640 1 librados: setting wanted keys 2023-07-29T13:28:22.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.276+0000 7f7d381b9640 1 librados: calling monclient init 2023-07-29T13:28:22.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.276+0000 7f7d381b9640 10 monclient: init 2023-07-29T13:28:22.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:22.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T13:28:22.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d300b74a0 0x7f7d300ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:22.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 10 monclient(hunting): picked mon.noname-a con 0x7f7d300b74a0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:22.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3013e4f0 0x7f7d301d5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:22.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 10 monclient(hunting): picked mon.noname-b con 0x7f7d3013e4f0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:22.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7d301445e0 0x7f7d300ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:22.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 10 monclient(hunting): picked mon.noname-c con 0x7f7d301445e0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:22.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:22.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301d6420 con 0x7f7d300b74a0 2023-07-29T13:28:22.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301d6100 con 0x7f7d3013e4f0 2023-07-29T13:28:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301d3770 con 0x7f7d301445e0 2023-07-29T13:28:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d381b9640 10 monclient(hunting): authenticate will time out at 1424.410383s 2023-07-29T13:28:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d35f55640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d300b74a0 0x7f7d300ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d35754640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3013e4f0 0x7f7d301d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.277+0000 7f7d36756640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7d301445e0 0x7f7d300ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35f55640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d300b74a0 0x7f7d300ac540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33138/0 (socket says 172.21.15.40:33138) 2023-07-29T13:28:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35f55640 1 -- 172.21.15.40:0/3448604305 learned_addr learned my addr 172.21.15.40:0/3448604305 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35f55640 10 monclient(hunting): get_auth_request con 0x7f7d300b74a0 auth_method 0 2023-07-29T13:28:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35f55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35f55640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35f55640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35754640 10 monclient(hunting): get_auth_request con 0x7f7d3013e4f0 auth_method 0 2023-07-29T13:28:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35754640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35754640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35754640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d36756640 10 monclient(hunting): get_auth_request con 0x7f7d301445e0 auth_method 0 2023-07-29T13:28:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d36756640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d36756640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d36756640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35754640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35754640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35754640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35f55640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35f55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d35f55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d36756640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.278+0000 7f7d36756640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.279+0000 7f7d36756640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.281+0000 7f7d35754640 10 monclient(hunting): handle_auth_done global_id 4354 payload 824 2023-07-29T13:28:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.281+0000 7f7d35754640 1 -- 172.21.15.40:0/3448604305 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7d301445e0 msgr2=0x7f7d300ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.281+0000 7f7d35754640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7d301445e0 0x7f7d300ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.281+0000 7f7d35754640 1 -- 172.21.15.40:0/3448604305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d300b74a0 msgr2=0x7f7d300ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.281+0000 7f7d35754640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d300b74a0 0x7f7d300ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.281+0000 7f7d35754640 10 monclient: _finish_hunting 0 2023-07-29T13:28:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.281+0000 7f7d35754640 1 monclient: found mon.noname-b 2023-07-29T13:28:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.281+0000 7f7d35754640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.281+0000 7f7d35754640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.281+0000 7f7d35754640 1 -- 172.21.15.40:0/3448604305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20009b30 con 0x7f7d3013e4f0 2023-07-29T13:28:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.284+0000 7f7d35754640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3013e4f0 0x7f7d301d5bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400e850 tx=0x7f7d2400ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.284+0000 7f7d2e7fc640 10 client.?.objecter ms_handle_connect 0x7f7d3013e4f0 2023-07-29T13:28:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.284+0000 7f7d2e7fc640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.287+0000 7f7d2e7fc640 1 -- 172.21.15.40:0/3448604305 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240095b0 con 0x7f7d3013e4f0 2023-07-29T13:28:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 10 monclient: dump: 2023-07-29T13:28:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:22.303 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 10 monclient: _finish_auth 0 2023-07-29T13:28:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 10 monclient: _check_auth_tickets 2023-07-29T13:28:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 1 -- 172.21.15.40:0/3448604305 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2400c940 con 0x7f7d3013e4f0 2023-07-29T13:28:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 1 -- 172.21.15.40:0/3448604305 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400ccd0 con 0x7f7d3013e4f0 2023-07-29T13:28:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2e7fc640 10 monclient: dump: 2023-07-29T13:28:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:22.305 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.305 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.305 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:22.305 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.305 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:22.306 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d381b9640 5 monclient: authenticate success, global_id 4354 2023-07-29T13:28:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d381b9640 10 monclient: _renew_subs 2023-07-29T13:28:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d381b9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d2ffff640 4 set_mon_vals no callback set 2023-07-29T13:28:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d381b9640 1 -- 172.21.15.40:0/3448604305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d300ad0f0 con 0x7f7d3013e4f0 2023-07-29T13:28:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d381b9640 10 client.4354.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d381b9640 10 monclient: _renew_subs 2023-07-29T13:28:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d381b9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d381b9640 1 -- 172.21.15.40:0/3448604305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d301d6a20 con 0x7f7d3013e4f0 2023-07-29T13:28:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.288+0000 7f7d381b9640 1 librados: init done 2023-07-29T13:28:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.289+0000 7f7d381b9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.289+0000 7f7d381b9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:22.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.289+0000 7f7d381b9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:22.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.289+0000 7f7d381b9640 1 -- 172.21.15.40:0/3448604305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3004e000 con 0x7f7d3013e4f0 2023-07-29T13:28:22.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.309+0000 7f7d2e7fc640 1 -- 172.21.15.40:0/3448604305 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f7d24013470 con 0x7f7d3013e4f0 2023-07-29T13:28:22.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.310+0000 7f7d2e7fc640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7d1c06bf90 0x7f7d1c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:22.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.310+0000 7f7d35f55640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7d1c06bf90 0x7f7d1c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:22.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.310+0000 7f7d35f55640 10 monclient: get_auth_request con 0x7f7d1c06bf90 auth_method 0 2023-07-29T13:28:22.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.310+0000 7f7d2e7fc640 10 client.4354.objecter ms_handle_connect 0x7f7d1c06bf90 2023-07-29T13:28:22.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.310+0000 7f7d35f55640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7d1c06bf90 0x7f7d1c06e440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d280018f0 tx=0x7f7d28009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.549+0000 7f7d2e7fc640 1 -- 172.21.15.40:0/3448604305 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7d2400ce30 con 0x7f7d3013e4f0 2023-07-29T13:28:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.549+0000 7f7d2e7fc640 10 client.4354.objecter ms_dispatch 0x7f7d30000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.549+0000 7f7d2e7fc640 3 client.4354.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.549+0000 7f7d2e7fc640 3 client.4354.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.549+0000 7f7d2e7fc640 20 client.4354.objecter dump_active .. 0 homeless 2023-07-29T13:28:22.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.560+0000 7f7d2e7fc640 1 -- 172.21.15.40:0/3448604305 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f7d24054860 con 0x7f7d3013e4f0 2023-07-29T13:28:22.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.560+0000 7f7d2e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.560+0000 7f7d2e7fc640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:22.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.738+0000 7f7d381b9640 1 -- 172.21.15.40:0/3448604305 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d300aa750 con 0x7f7d1c06bf90 2023-07-29T13:28:22.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.740+0000 7f7d2e7fc640 1 -- 172.21.15.40:0/3448604305 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47959 (secure 0 0 0) 0x7f7d280015e0 con 0x7f7d1c06bf90 2023-07-29T13:28:22.742 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:28:22.742 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-29T13:28:22.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 10 librados: watch_flush enter 2023-07-29T13:28:22.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 10 librados: watch_flush exit 2023-07-29T13:28:22.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 20 client.4354.objecter shutdown clearing up homeless session... 2023-07-29T13:28:22.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 10 client.4354.objecter successfully canceled tick 2023-07-29T13:28:22.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 -- 172.21.15.40:0/3448604305 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7d1c06bf90 msgr2=0x7f7d1c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7d1c06bf90 0x7f7d1c06e440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d280018f0 tx=0x7f7d28009040 comp rx=0 tx=0).stop 2023-07-29T13:28:22.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 10 monclient: shutdown 2023-07-29T13:28:22.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:22.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 -- 172.21.15.40:0/3448604305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3013e4f0 msgr2=0x7f7d301d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3013e4f0 0x7f7d301d5bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400e850 tx=0x7f7d2400ed10 comp rx=0 tx=0).stop 2023-07-29T13:28:22.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 -- 172.21.15.40:0/3448604305 shutdown_connections 2023-07-29T13:28:22.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7d1c06bf90 0x7f7d1c06e440 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3013e4f0 0x7f7d301d5bc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7d301445e0 0x7f7d300ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 --2- 172.21.15.40:0/3448604305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d300b74a0 0x7f7d300ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 -- 172.21.15.40:0/3448604305 >> 172.21.15.40:0/3448604305 conn(0x7f7d30065b00 msgr2=0x7f7d3013a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:22.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 -- 172.21.15.40:0/3448604305 shutdown_connections 2023-07-29T13:28:22.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 -- 172.21.15.40:0/3448604305 wait complete. 2023-07-29T13:28:22.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.742+0000 7f7d381b9640 1 librados: shutdown 2023-07-29T13:28:22.754 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-07-29T13:28:21.016427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218256,"kb_used_data":2640,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756464,"statfs":{"total":773094113280,"available":772870619136,"internally_reserved":0,"allocated":2703360,"data_stored":1441616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.137619"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.038384+0000","last_change":"2023-07-29T13:28:14.694035+0000","last_active":"2023-07-29T13:28:15.038384+0000","last_peered":"2023-07-29T13:28:15.038384+0000","last_clean":"2023-07-29T13:28:15.038384+0000","last_became_active":"2023-07-29T13:28:14.688540+0000","last_became_peered":"2023-07-29T13:28:14.688540+0000","last_unstale":"2023-07-29T13:28:15.038384+0000","last_undegraded":"2023-07-29T13:28:15.038384+0000","last_fullsized":"2023-07-29T13:28:15.038384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:31:19.269462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.828590+0000","last_change":"2023-07-29T13:28:15.118143+0000","last_active":"2023-07-29T13:28:15.828590+0000","last_peered":"2023-07-29T13:28:15.828590+0000","last_clean":"2023-07-29T13:28:15.828590+0000","last_became_active":"2023-07-29T13:28:15.089605+0000","last_became_peered":"2023-07-29T13:28:15.089605+0000","last_unstale":"2023-07-29T13:28:15.828590+0000","last_undegraded":"2023-07-29T13:28:15.828590+0000","last_fullsized":"2023-07-29T13:28:15.828590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:00:18.108603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.047826+0000","last_change":"2023-07-29T13:28:14.434312+0000","last_active":"2023-07-29T13:28:15.047826+0000","last_peered":"2023-07-29T13:28:15.047826+0000","last_clean":"2023-07-29T13:28:15.047826+0000","last_became_active":"2023-07-29T13:28:14.423044+0000","last_became_peered":"2023-07-29T13:28:14.423044+0000","last_unstale":"2023-07-29T13:28:15.047826+0000","last_undegraded":"2023-07-29T13:28:15.047826+0000","last_fullsized":"2023-07-29T13:28:15.047826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:56:27.865715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.042026+0000","last_change":"2023-07-29T13:28:14.686642+0000","last_active":"2023-07-29T13:28:15.042026+0000","last_peered":"2023-07-29T13:28:15.042026+0000","last_clean":"2023-07-29T13:28:15.042026+0000","last_became_active":"2023-07-29T13:28:14.659427+0000","last_became_peered":"2023-07-29T13:28:14.659427+0000","last_unstale":"2023-07-29T13:28:15.042026+0000","last_undegraded":"2023-07-29T13:28:15.042026+0000","last_fullsized":"2023-07-29T13:28:15.042026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:24:15.162936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.029326+0000","last_change":"2023-07-29T13:28:14.652586+0000","last_active":"2023-07-29T13:28:15.029326+0000","last_peered":"2023-07-29T13:28:15.029326+0000","last_clean":"2023-07-29T13:28:15.029326+0000","last_became_active":"2023-07-29T13:28:14.642023+0000","last_became_peered":"2023-07-29T13:28:14.642023+0000","last_unstale":"2023-07-29T13:28:15.029326+0000","last_undegraded":"2023-07-29T13:28:15.029326+0000","last_fullsized":"2023-07-29T13:28:15.029326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:25:23.334956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:14.889983+0000","last_change":"2023-07-29T13:28:14.256724+0000","last_active":"2023-07-29T13:28:14.889983+0000","last_peered":"2023-07-29T13:28:14.889983+0000","last_clean":"2023-07-29T13:28:14.889983+0000","last_became_active":"2023-07-29T13:28:14.228272+0000","last_became_peered":"2023-07-29T13:28:14.228272+0000","last_unstale":"2023-07-29T13:28:14.889983+0000","last_undegraded":"2023-07-29T13:28:14.889983+0000","last_fullsized":"2023-07-29T13:28:14.889983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:24:42.037027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.041783+0000","last_change":"2023-07-29T13:28:14.384771+0000","last_active":"2023-07-29T13:28:15.041783+0000","last_peered":"2023-07-29T13:28:15.041783+0000","last_clean":"2023-07-29T13:28:15.041783+0000","last_became_active":"2023-07-29T13:28:14.353364+0000","last_became_peered":"2023-07-29T13:28:14.353364+0000","last_unstale":"2023-07-29T13:28:15.041783+0000","last_undegraded":"2023-07-29T13:28:15.041783+0000","last_fullsized":"2023-07-29T13:28:15.041783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:28:57.053189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.034836+0000","last_change":"2023-07-29T13:28:14.713485+0000","last_active":"2023-07-29T13:28:15.034836+0000","last_peered":"2023-07-29T13:28:15.034836+0000","last_clean":"2023-07-29T13:28:15.034836+0000","last_became_active":"2023-07-29T13:28:14.708042+0000","last_became_peered":"2023-07-29T13:28:14.708042+0000","last_unstale":"2023-07-29T13:28:15.034836+0000","last_undegraded":"2023-07-29T13:28:15.034836+0000","last_fullsized":"2023-07-29T13:28:15.034836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:25:56.525512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"21'64","reported_seq":103,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-29T13:28:15.003696+0000","last_change":"2023-07-29T13:28:10.113325+0000","last_active":"2023-07-29T13:28:15.003696+0000","last_peered":"2023-07-29T13:28:15.003696+0000","last_clean":"2023-07-29T13:28:15.003696+0000","last_became_active":"2023-07-29T13:28:10.081018+0000","last_became_peered":"2023-07-29T13:28:10.081018+0000","last_unstale":"2023-07-29T13:28:15.003696+0000","last_undegraded":"2023-07-29T13:28:15.003696+0000","last_fullsized":"2023-07-29T13:28:15.003696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_clean_scrub_stamp":"2023-07-29T13:28:06.434639+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:30:49.511284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":17,"seq":73014444038,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.481}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.553000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.323}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.346}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.866}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.579000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":25.216999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.2720000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.7509999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.811999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.3889999999999993}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.686999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.103999999999999}]}]},{"osd":1,"up_from":17,"seq":73014444039,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":478582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.928000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.404}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":31.324999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.356}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":37.991999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.266}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.603000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444039,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.337}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.655999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.238}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":27.369}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.73}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.853000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.436}]}]},{"osd":4,"up_from":16,"seq":68719476741,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.238}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.239000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.9130000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.244}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.343999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.911999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.378}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4019999999999992}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.190000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.834}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.776}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.959}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.588000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":478582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5969999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.105}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.7940000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.31}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.083}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.664}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.018999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.3689999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.223000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.343999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.756}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.170999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.978999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-29T13:28:22.755 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-29T13:28:22.755 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-29T13:28:22.756 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-07-29T13:28:22.756 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-29T13:28:22.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.861+0000 7f83c4bcc640 10 monclient: get_monmap_and_config 2023-07-29T13:28:22.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.861+0000 7f83c4bcc640 10 monclient: build_initial_monmap 2023-07-29T13:28:22.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.861+0000 7f83c4bcc640 10 monclient: monmap: 2023-07-29T13:28:22.863 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:22.863 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:22.863 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:28:22.862465+0000 2023-07-29T13:28:22.863 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:28:22.862465+0000 2023-07-29T13:28:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:28:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:28:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:28:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:28:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:28:22.865 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 1 Processor -- start 2023-07-29T13:28:22.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 1 -- start start 2023-07-29T13:28:22.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 10 monclient: init 2023-07-29T13:28:22.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:22.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2023-07-29T13:28:22.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f83c00b3ae0 0x7f83c013dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:22.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 10 monclient(hunting): picked mon.noname-c con 0x7f83c00b3ae0 addr v2:172.21.15.45:3300/0 2023-07-29T13:28:22.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f83c00a8910 addr v1:172.21.15.40:6789/0 2023-07-29T13:28:22.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f83c0141320 addr v1:172.21.15.45:6789/0 2023-07-29T13:28:22.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c00b3eb0 con 0x7f83c00a8910 2023-07-29T13:28:22.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c00a9950 con 0x7f83c0141320 2023-07-29T13:28:22.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:22.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c00b0f10 con 0x7f83c00b3ae0 2023-07-29T13:28:22.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:22.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.862+0000 7f83c4bcc640 10 monclient(hunting): authenticate will time out at 1424.995779s 2023-07-29T13:28:22.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.863+0000 7f83be59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f83c00b3ae0 0x7f83c013dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:22.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.863+0000 7f83bed9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f83c0141320 0x7f83c014b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.40:46586/0 (socket says 172.21.15.40:46586) 2023-07-29T13:28:22.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.863+0000 7f83bed9d640 1 -- 172.21.15.40:0/2030749917 learned_addr learned my addr 172.21.15.40:0/2030749917 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T13:28:22.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.864+0000 7f83be59c640 10 monclient(hunting): get_auth_request con 0x7f83c00b3ae0 auth_method 0 2023-07-29T13:28:22.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.864+0000 7f83be59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:22.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.864+0000 7f83be59c640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:22.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.864+0000 7f83be59c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:22.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.865+0000 7f83be59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:22.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.865+0000 7f83be59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:22.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.865+0000 7f83be59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:22.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.866+0000 7f83be59c640 10 monclient(hunting): handle_auth_done global_id 4357 payload 274 2023-07-29T13:28:22.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.866+0000 7f83be59c640 1 -- 172.21.15.40:0/2030749917 >> v1:172.21.15.45:6789/0 conn(0x7f83c0141320 legacy=0x7f83c014b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.866+0000 7f83be59c640 1 -- 172.21.15.40:0/2030749917 >> v1:172.21.15.40:6789/0 conn(0x7f83c00a8910 legacy=0x7f83c0140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:28:22.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.866+0000 7f83be59c640 10 monclient: _finish_hunting 0 2023-07-29T13:28:22.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.866+0000 7f83be59c640 1 monclient: found mon.noname-c 2023-07-29T13:28:22.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.866+0000 7f83be59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:22.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.866+0000 7f83be59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T13:28:22.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.866+0000 7f83be59c640 1 -- 172.21.15.40:0/2030749917 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c00a9c30 con 0x7f83c00b3ae0 2023-07-29T13:28:22.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.871+0000 7f83be59c640 1 --2- 172.21.15.40:0/2030749917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c00b3ae0 0x7f83c013dc90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f83b00097a0 tx=0x7f83b002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=811b211033c50e99 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:22.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 1 -- 172.21.15.40:0/2030749917 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b002e070 con 0x7f83c00b3ae0 2023-07-29T13:28:22.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:22.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T13:28:22.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 10 monclient: dump: 2023-07-29T13:28:22.878 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:22.878 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:22.878 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.878 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.878 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:22.878 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.879 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:22.879 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:22.879 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:22.879 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 10 monclient: _finish_auth 0 2023-07-29T13:28:22.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 10 monclient: _check_auth_tickets 2023-07-29T13:28:22.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:22.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 1 -- 172.21.15.40:0/2030749917 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b00315a0 con 0x7f83c00b3ae0 2023-07-29T13:28:22.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:22.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 1 -- 172.21.15.40:0/2030749917 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0031930 con 0x7f83c00b3ae0 2023-07-29T13:28:22.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:22.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:22.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83bdd9b640 10 monclient: dump: 2023-07-29T13:28:22.880 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:22.880 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:22.881 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.881 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.881 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:22.881 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.881 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:22.881 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:22.881 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:22.881 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83c4bcc640 5 monclient: authenticate success, global_id 4357 2023-07-29T13:28:22.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83c4bcc640 10 monclient: get_monmap_and_config success 2023-07-29T13:28:22.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.875+0000 7f83c4bcc640 4 set_mon_vals no callback set 2023-07-29T13:28:22.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.876+0000 7f83c4bcc640 10 monclient: shutdown 2023-07-29T13:28:22.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.876+0000 7f83c4bcc640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:22.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.876+0000 7f83c4bcc640 1 -- 172.21.15.40:0/2030749917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c00b3ae0 msgr2=0x7f83c013dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.876+0000 7f83c4bcc640 1 --2- 172.21.15.40:0/2030749917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c00b3ae0 0x7f83c013dc90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f83b00097a0 tx=0x7f83b002b840 comp rx=0 tx=0).stop 2023-07-29T13:28:22.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.876+0000 7f83c4bcc640 1 -- 172.21.15.40:0/2030749917 shutdown_connections 2023-07-29T13:28:22.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.876+0000 7f83c4bcc640 1 --2- 172.21.15.40:0/2030749917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c00b3ae0 0x7f83c013dc90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.876+0000 7f83c4bcc640 1 -- 172.21.15.40:0/2030749917 >> 172.21.15.40:0/2030749917 conn(0x7f83c0070fc0 msgr2=0x7f83c00713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:22.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.876+0000 7f83c4bcc640 1 -- 172.21.15.40:0/2030749917 shutdown_connections 2023-07-29T13:28:22.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.876+0000 7f83c4bcc640 1 -- 172.21.15.40:0/2030749917 wait complete. 2023-07-29T13:28:22.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.877+0000 7f83c4bcc640 10 monclient: build_initial_monmap 2023-07-29T13:28:22.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.877+0000 7f83c4bcc640 10 monclient: monmap: 2023-07-29T13:28:22.886 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T13:28:22.886 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:28:22.886 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T13:28:22.886 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T13:28:22.887 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T13:28:22.887 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.887 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:28:22.887 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:28:22.887 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:28:22.887 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.877+0000 7f83c4bcc640 1 librados: starting msgr at 2023-07-29T13:28:22.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.877+0000 7f83c4bcc640 1 librados: starting objecter 2023-07-29T13:28:22.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.877+0000 7f83c4bcc640 1 Processor -- start 2023-07-29T13:28:22.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.877+0000 7f83c4bcc640 1 -- start start 2023-07-29T13:28:22.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.877+0000 7f83c4bcc640 1 librados: setting wanted keys 2023-07-29T13:28:22.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.877+0000 7f83c4bcc640 1 librados: calling monclient init 2023-07-29T13:28:22.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.877+0000 7f83c4bcc640 10 monclient: init 2023-07-29T13:28:22.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 10 monclient: _reopen_session rank -1 2023-07-29T13:28:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 10 monclient: _add_conns ranks=[1,0,2] 2023-07-29T13:28:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c00a8910 0x7f83c0144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 10 monclient(hunting): picked mon.noname-b con 0x7f83c00a8910 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:28:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c00b3ae0 0x7f83c00ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 10 monclient(hunting): picked mon.noname-a con 0x7f83c00b3ae0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:28:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f83c0141320 0x7f83c013a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 10 monclient(hunting): picked mon.noname-c con 0x7f83c0141320 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:28:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00ac070 con 0x7f83c00b3ae0 2023-07-29T13:28:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00acd20 con 0x7f83c00a8910 2023-07-29T13:28:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 10 monclient(hunting): start opening mon connection 2023-07-29T13:28:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00acfd0 con 0x7f83c0141320 2023-07-29T13:28:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 10 monclient(hunting): _renew_subs 2023-07-29T13:28:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83c4bcc640 10 monclient(hunting): authenticate will time out at 1425.011453s 2023-07-29T13:28:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83be59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c00a8910 0x7f83c0144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c00b3ae0 0x7f83c00ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.878+0000 7f83bed9d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f83c0141320 0x7f83c013a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83be59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c00a8910 0x7f83c0144820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:59534/0 (socket says 172.21.15.40:59534) 2023-07-29T13:28:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83be59c640 1 -- 172.21.15.40:0/1495380099 learned_addr learned my addr 172.21.15.40:0/1495380099 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:28:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83be59c640 10 monclient(hunting): get_auth_request con 0x7f83c00a8910 auth_method 0 2023-07-29T13:28:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83be59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83be59c640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83be59c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83b7fff640 10 monclient(hunting): get_auth_request con 0x7f83c00b3ae0 auth_method 0 2023-07-29T13:28:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83b7fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83b7fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83bed9d640 10 monclient(hunting): get_auth_request con 0x7f83c0141320 auth_method 0 2023-07-29T13:28:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83bed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:28:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83bed9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:28:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83bed9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:28:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83be59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83be59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83be59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83bed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:28:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83bed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:28:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.879+0000 7f83bed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:28:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.882+0000 7f83be59c640 10 monclient(hunting): handle_auth_done global_id 4360 payload 824 2023-07-29T13:28:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.882+0000 7f83be59c640 1 -- 172.21.15.40:0/1495380099 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f83c0141320 msgr2=0x7f83c013a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.882+0000 7f83be59c640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f83c0141320 0x7f83c013a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.882+0000 7f83be59c640 1 -- 172.21.15.40:0/1495380099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c00b3ae0 msgr2=0x7f83c00ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.882+0000 7f83be59c640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c00b3ae0 0x7f83c00ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.882+0000 7f83be59c640 10 monclient: _finish_hunting 0 2023-07-29T13:28:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.882+0000 7f83be59c640 1 monclient: found mon.noname-b 2023-07-29T13:28:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.882+0000 7f83be59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:28:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.882+0000 7f83be59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:28:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.882+0000 7f83be59c640 1 -- 172.21.15.40:0/1495380099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b0009400 con 0x7f83c00a8910 2023-07-29T13:28:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.884+0000 7f83be59c640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c00a8910 0x7f83c0144820 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f83b0000c00 tx=0x7f83b00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.884+0000 7f83b6ffd640 10 client.?.objecter ms_handle_connect 0x7f83c00a8910 2023-07-29T13:28:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.884+0000 7f83b6ffd640 10 client.?.objecter resend_mon_ops 2023-07-29T13:28:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 1 -- 172.21.15.40:0/1495380099 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0032900 con 0x7f83c00a8910 2023-07-29T13:28:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:28:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 10 monclient: dump: 2023-07-29T13:28:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:22.899 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 10 monclient: _finish_auth 0 2023-07-29T13:28:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 10 monclient: _check_auth_tickets 2023-07-29T13:28:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:28:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 1 -- 172.21.15.40:0/1495380099 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b0032a80 con 0x7f83c00a8910 2023-07-29T13:28:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:28:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 1 -- 172.21.15.40:0/1495380099 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0032e30 con 0x7f83c00a8910 2023-07-29T13:28:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83c4bcc640 5 monclient: authenticate success, global_id 4360 2023-07-29T13:28:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83c4bcc640 10 monclient: _renew_subs 2023-07-29T13:28:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83c4bcc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83c4bcc640 1 -- 172.21.15.40:0/1495380099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c00ab350 con 0x7f83c00a8910 2023-07-29T13:28:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83bd59a640 4 set_mon_vals no callback set 2023-07-29T13:28:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:28:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:28:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83b6ffd640 10 monclient: dump: 2023-07-29T13:28:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T13:28:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:28:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:28:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T13:28:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T13:28:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:28:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:28:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:28:22.902 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T13:28:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83c4bcc640 10 client.4360.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:28:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83c4bcc640 10 monclient: _renew_subs 2023-07-29T13:28:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83c4bcc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83c4bcc640 1 -- 172.21.15.40:0/1495380099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c013a980 con 0x7f83c00a8910 2023-07-29T13:28:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.888+0000 7f83c4bcc640 1 librados: init done 2023-07-29T13:28:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.889+0000 7f83c4bcc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.889+0000 7f83c4bcc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.889+0000 7f83c4bcc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:22.889+0000 7f83c4bcc640 1 -- 172.21.15.40:0/1495380099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c00ad140 con 0x7f83c00a8910 2023-07-29T13:28:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.149+0000 7f83b6ffd640 1 -- 172.21.15.40:0/1495380099 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f83b004e470 con 0x7f83c00a8910 2023-07-29T13:28:23.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.150+0000 7f83b6ffd640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f83a006bfb0 0x7f83a006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:28:23.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.150+0000 7f83b7fff640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f83a006bfb0 0x7f83a006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:28:23.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.150+0000 7f83b7fff640 10 monclient: get_auth_request con 0x7f83a006bfb0 auth_method 0 2023-07-29T13:28:23.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.151+0000 7f83b6ffd640 10 client.4360.objecter ms_handle_connect 0x7f83a006bfb0 2023-07-29T13:28:23.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.151+0000 7f83b7fff640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f83a006bfb0 0x7f83a006e460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83c00abcb0 tx=0x7f83a8009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:28:23.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.152+0000 7f83b6ffd640 1 -- 172.21.15.40:0/1495380099 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f83b007db60 con 0x7f83c00a8910 2023-07-29T13:28:23.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.152+0000 7f83b6ffd640 10 client.4360.objecter ms_dispatch 0x7f83c0000b90 osd_map(24..24 src has 1..24) v4 2023-07-29T13:28:23.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.152+0000 7f83b6ffd640 3 client.4360.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:28:23.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.152+0000 7f83b6ffd640 3 client.4360.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:28:23.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.152+0000 7f83b6ffd640 20 client.4360.objecter dump_active .. 0 homeless 2023-07-29T13:28:23.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.162+0000 7f83b6ffd640 1 -- 172.21.15.40:0/1495380099 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f83b0075f90 con 0x7f83c00a8910 2023-07-29T13:28:23.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.162+0000 7f83b6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T13:28:23.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.162+0000 7f83b6ffd640 10 monclient: _finish_command 1 = system:0 2023-07-29T13:28:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.378+0000 7f83c4bcc640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2023-07-29T13:28:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.378+0000 7f83c4bcc640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2023-07-29T13:28:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.378+0000 7f83c4bcc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:28:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.378+0000 7f83c4bcc640 1 -- 172.21.15.40:0/1495380099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f83c0067860 con 0x7f83c00a8910 2023-07-29T13:28:23.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.385+0000 7f83b6ffd640 1 -- 172.21.15.40:0/1495380099 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f83b00795b0 con 0x7f83c00a8910 2023-07-29T13:28:23.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.385+0000 7f83b6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2023-07-29T13:28:23.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.385+0000 7f83b6ffd640 10 monclient: _finish_command 2 = system:0 2023-07-29T13:28:23.387 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T13:28:23.387 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-29T13:28:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 10 librados: watch_flush enter 2023-07-29T13:28:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 10 librados: watch_flush exit 2023-07-29T13:28:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 20 client.4360.objecter shutdown clearing up homeless session... 2023-07-29T13:28:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 10 client.4360.objecter successfully canceled tick 2023-07-29T13:28:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 -- 172.21.15.40:0/1495380099 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f83a006bfb0 msgr2=0x7f83a006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f83a006bfb0 0x7f83a006e460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83c00abcb0 tx=0x7f83a8009040 comp rx=0 tx=0).stop 2023-07-29T13:28:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 10 monclient: shutdown 2023-07-29T13:28:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:28:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 -- 172.21.15.40:0/1495380099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c00a8910 msgr2=0x7f83c0144820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:28:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c00a8910 0x7f83c0144820 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f83b0000c00 tx=0x7f83b00046c0 comp rx=0 tx=0).stop 2023-07-29T13:28:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 -- 172.21.15.40:0/1495380099 shutdown_connections 2023-07-29T13:28:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f83a006bfb0 0x7f83a006e460 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f83c0141320 0x7f83c013a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c00b3ae0 0x7f83c00ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 --2- 172.21.15.40:0/1495380099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c00a8910 0x7f83c0144820 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:28:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 -- 172.21.15.40:0/1495380099 >> 172.21.15.40:0/1495380099 conn(0x7f83c0070fc0 msgr2=0x7f83c00b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:28:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 -- 172.21.15.40:0/1495380099 shutdown_connections 2023-07-29T13:28:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 -- 172.21.15.40:0/1495380099 wait complete. 2023-07-29T13:28:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T13:28:23.387+0000 7f83c4bcc640 1 librados: shutdown 2023-07-29T13:28:23.399 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-07-29T13:28:23.400 INFO:teuthology.run_tasks:Running task workunit... 2023-07-29T13:28:23.418 INFO:tasks.workunit:Pulling workunits from ref 0484097b0572e96f42cf6402cbe0ac8dcb046577 2023-07-29T13:28:23.418 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-07-29T13:28:23.419 DEBUG:teuthology.orchestra.run.smithi040:> stat -- /home/ubuntu/cephtest/mnt.0 2023-07-29T13:28:23.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-29T13:28:23.434 INFO:teuthology.orchestra.run.smithi040.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-07-29T13:28:23.434 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-07-29T13:28:23.489 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-07-29T13:28:23.490 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-07-29T13:28:23.545 INFO:tasks.workunit:timeout=6h 2023-07-29T13:28:23.545 INFO:tasks.workunit:cleanup=True 2023-07-29T13:28:23.545 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 0484097b0572e96f42cf6402cbe0ac8dcb046577 2023-07-29T13:28:23.600 INFO:tasks.workunit.client.0.smithi040.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-07-29T13:28:25.255 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:01:20.589 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:25.312 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:01:20.646 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:25.326 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:01:20.660 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:25.331 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:01:20.615 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:25.382 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:01:20.717 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:25.386 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:01:20.670 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:25.568 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:01:20.902 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:25.639 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:01:20.923 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:25.648 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:01:20.982 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:45.303 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:01:40.637 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:45.308 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:01:40.642 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:45.360 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:01:40.644 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:45.363 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:01:40.648 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:45.366 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:01:40.700 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:45.387 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:01:40.721 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:45.399 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:01:40.734 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:45.400 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:01:40.684 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:28:45.452 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:01:40.787 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:05.371 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:02:00.705 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:05.375 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:02:00.709 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:05.397 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:02:00.731 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:05.428 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:02:00.712 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:05.429 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:02:00.713 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:05.443 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:02:00.727 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:05.453 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:02:00.787 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:05.457 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:02:00.791 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:05.479 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:02:00.813 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:11.866 INFO:tasks.workunit.client.0.smithi040.stderr:Updating files: 97% (11340/11620) Updating files: 98% (11388/11620) Updating files: 99% (11504/11620) Updating files: 100% (11620/11620) Updating files: 100% (11620/11620), done. 2023-07-29T13:29:12.682 INFO:tasks.workunit.client.0.smithi040.stderr:Note: switching to '0484097b0572e96f42cf6402cbe0ac8dcb046577'. 2023-07-29T13:29:12.683 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-29T13:29:12.683 INFO:tasks.workunit.client.0.smithi040.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-07-29T13:29:12.683 INFO:tasks.workunit.client.0.smithi040.stderr:changes and commit them, and you can discard any commits you make in this 2023-07-29T13:29:12.683 INFO:tasks.workunit.client.0.smithi040.stderr:state without impacting any branches by switching back to a branch. 2023-07-29T13:29:12.683 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-29T13:29:12.684 INFO:tasks.workunit.client.0.smithi040.stderr:If you want to create a new branch to retain commits you create, you may 2023-07-29T13:29:12.684 INFO:tasks.workunit.client.0.smithi040.stderr:do so (now or later) by using -c with the switch command. Example: 2023-07-29T13:29:12.684 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-29T13:29:12.684 INFO:tasks.workunit.client.0.smithi040.stderr: git switch -c 2023-07-29T13:29:12.684 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-29T13:29:12.684 INFO:tasks.workunit.client.0.smithi040.stderr:Or undo this operation with: 2023-07-29T13:29:12.684 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-29T13:29:12.684 INFO:tasks.workunit.client.0.smithi040.stderr: git switch - 2023-07-29T13:29:12.685 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-29T13:29:12.685 INFO:tasks.workunit.client.0.smithi040.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-07-29T13:29:12.685 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-29T13:29:12.685 INFO:tasks.workunit.client.0.smithi040.stderr:HEAD is now at 0484097b057 Merge pull request #52618 from vshankar/tr-62147 2023-07-29T13:29:12.686 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-07-29T13:29:12.749 INFO:tasks.workunit.client.0.smithi040.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-07-29T13:29:12.751 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-29T13:29:12.751 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-07-29T13:29:13.318 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-07-29T13:29:13.380 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-07-29T13:29:13.421 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-29T13:29:13.422 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-29T13:29:13.423 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-07-29T13:29:13.463 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-29T13:29:13.466 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-29T13:29:13.466 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-07-29T13:29:13.522 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2023-07-29T13:29:13.524 INFO:tasks.workunit:Running workunit rados/test.sh... 2023-07-29T13:29:13.524 DEBUG:teuthology.orchestra.run.smithi040:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0484097b0572e96f42cf6402cbe0ac8dcb046577 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2023-07-29T13:29:13.582 INFO:tasks.workunit.client.0.smithi040.stderr:+ parallel=1 2023-07-29T13:29:13.582 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' '' = --serial ']' 2023-07-29T13:29:13.583 INFO:tasks.workunit.client.0.smithi040.stderr:+ color= 2023-07-29T13:29:13.583 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -t 1 ']' 2023-07-29T13:29:13.583 INFO:tasks.workunit.client.0.smithi040.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2023-07-29T13:29:13.583 INFO:tasks.workunit.client.0.smithi040.stderr:+ declare -A pids 2023-07-29T13:29:13.583 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.584 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.584 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_aio 2023-07-29T13:29:13.584 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_aio' 2023-07-29T13:29:13.585 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_aio 2023-07-29T13:29:13.585 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.585 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_aio 2023-07-29T13:29:13.585 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90635 2023-07-29T13:29:13.586 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_aio on pid 90635' 2023-07-29T13:29:13.586 INFO:tasks.workunit.client.0.smithi040.stdout:test api_aio on pid 90635 2023-07-29T13:29:13.588 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90635 2023-07-29T13:29:13.588 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2023-07-29T13:29:13.588 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.588 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.589 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_aio_pp 2023-07-29T13:29:13.589 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_aio_pp' 2023-07-29T13:29:13.589 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_aio_pp 2023-07-29T13:29:13.589 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.589 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.589 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.589 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.589 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.590 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.590 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.590 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.590 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.590 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.590 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.590 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.590 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.591 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.591 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.591 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.591 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.591 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.591 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.591 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.591 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.592 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.592 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.592 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.592 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.592 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.592 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_aio_pp 2023-07-29T13:29:13.592 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90642 2023-07-29T13:29:13.593 INFO:tasks.workunit.client.0.smithi040.stdout:test api_aio_pp on pid 90642 2023-07-29T13:29:13.595 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_aio_pp on pid 90642' 2023-07-29T13:29:13.595 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90642 2023-07-29T13:29:13.596 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.596 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.596 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2023-07-29T13:29:13.596 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_io 2023-07-29T13:29:13.596 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_io' 2023-07-29T13:29:13.596 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_io 2023-07-29T13:29:13.596 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.597 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.597 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.597 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.597 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.597 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.597 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.597 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.597 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.598 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.598 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.598 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.598 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.598 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.598 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.598 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.599 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.599 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.599 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.599 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.599 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.599 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.599 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.599 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.600 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.600 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.600 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.600 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.600 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.600 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.600 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.601 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.601 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.601 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.601 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.601 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.602 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.602 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.602 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.602 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.602 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.602 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.602 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.602 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.603 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.603 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.603 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_io 2023-07-29T13:29:13.603 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90651 2023-07-29T13:29:13.603 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_io on pid 90651' 2023-07-29T13:29:13.603 INFO:tasks.workunit.client.0.smithi040.stdout:test api_io on pid 90651 2023-07-29T13:29:13.608 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90651 2023-07-29T13:29:13.608 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.608 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.608 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2023-07-29T13:29:13.608 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_io_pp 2023-07-29T13:29:13.609 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_io_pp' 2023-07-29T13:29:13.609 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_io_pp 2023-07-29T13:29:13.609 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.609 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.609 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.609 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.609 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.609 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.610 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.610 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.610 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.610 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.610 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.610 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.610 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.611 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.611 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.611 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.611 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.611 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.611 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.611 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.611 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.612 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.612 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.612 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.612 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.612 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.612 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.612 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.612 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.613 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.613 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.613 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.613 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.613 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.613 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.613 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.614 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.614 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.614 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.614 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.614 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.614 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.614 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.614 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.615 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.615 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.615 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.615 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.615 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.615 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.615 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.615 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.616 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.616 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.616 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.616 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_io_pp 2023-07-29T13:29:13.616 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90662 2023-07-29T13:29:13.616 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_io_pp on pid 90662' 2023-07-29T13:29:13.617 INFO:tasks.workunit.client.0.smithi040.stdout:test api_io_pp on pid 90662 2023-07-29T13:29:13.620 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90662 2023-07-29T13:29:13.621 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.621 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.621 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2023-07-29T13:29:13.621 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_asio 2023-07-29T13:29:13.621 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_asio' 2023-07-29T13:29:13.621 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_asio 2023-07-29T13:29:13.621 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.622 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.622 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.622 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.622 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.622 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.622 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.622 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.623 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.623 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.623 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.623 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.623 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.623 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.623 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.623 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.624 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.624 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.624 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.624 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.624 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.624 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.625 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.625 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.625 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.625 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.625 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.625 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.625 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.625 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.626 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.626 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.626 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.626 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.626 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.626 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.626 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.626 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.627 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.627 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.627 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.627 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.627 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.627 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.627 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.627 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.628 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.628 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.628 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.628 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.628 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.628 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.628 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.629 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.629 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.629 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.629 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.629 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.629 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.629 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.629 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.630 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.630 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.630 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.630 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.630 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.630 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.630 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.631 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.631 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.631 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.631 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.632 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.632 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_asio 2023-07-29T13:29:13.632 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90676 2023-07-29T13:29:13.632 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_asio on pid 90676' 2023-07-29T13:29:13.633 INFO:tasks.workunit.client.0.smithi040.stdout:test api_asio on pid 90676 2023-07-29T13:29:13.640 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90676 2023-07-29T13:29:13.640 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2023-07-29T13:29:13.640 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.640 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.640 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_list 2023-07-29T13:29:13.640 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.641 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.641 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.641 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.641 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.641 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_list' 2023-07-29T13:29:13.641 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.641 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_list 2023-07-29T13:29:13.641 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.642 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.642 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.642 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.642 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.642 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.642 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.642 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.643 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.643 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.643 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.643 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.643 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.643 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.643 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.643 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.644 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.644 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.644 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.644 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.644 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.644 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.644 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.644 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.645 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.645 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.645 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.645 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.645 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.645 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.645 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.646 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.646 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.646 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.646 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.646 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.646 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.646 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.646 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.647 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.647 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.647 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.647 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.647 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.647 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.647 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.647 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.648 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.648 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.648 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.648 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.648 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.648 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.648 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.649 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.649 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.649 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.649 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.649 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.649 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.649 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.649 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.650 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.650 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.650 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.650 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.650 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.650 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.650 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.650 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.651 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.651 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.651 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.651 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.651 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.651 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.651 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.652 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.652 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.652 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.652 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.652 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.652 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.652 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.652 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.653 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.653 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.653 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.653 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.653 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.653 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.653 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.653 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.654 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.654 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_list 2023-07-29T13:29:13.654 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90635/exe 2023-07-29T13:29:13.654 INFO:tasks.workunit.client.0.smithi040.stdout:test api_list on pid 90693 2023-07-29T13:29:13.657 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90693 2023-07-29T13:29:13.657 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_list on pid 90693' 2023-07-29T13:29:13.658 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90693 2023-07-29T13:29:13.658 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.658 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.658 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2023-07-29T13:29:13.658 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_lock 2023-07-29T13:29:13.658 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_lock' 2023-07-29T13:29:13.658 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.659 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.659 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.659 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.659 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.659 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.659 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_lock 2023-07-29T13:29:13.659 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.659 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.660 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.660 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.660 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.660 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.660 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.660 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.660 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.661 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.661 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.661 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.661 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.661 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.661 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.662 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.662 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.662 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.662 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.662 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.662 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.662 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.662 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.663 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.663 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.663 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.663 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.663 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.663 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.663 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.663 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.664 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.664 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.664 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.665 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.665 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.665 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.665 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.665 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.665 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.665 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.665 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.666 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.666 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.666 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.666 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.666 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.666 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.666 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.666 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.667 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.667 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.667 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.667 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_aio 2023-07-29T13:29:13.667 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.667 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.667 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.667 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.668 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.668 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.668 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.668 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.668 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.668 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.668 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.669 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.669 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.669 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.669 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.669 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.669 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.669 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.670 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.670 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.670 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.670 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.670 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.670 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.670 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.670 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.671 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_aio: /' 2023-07-29T13:29:13.671 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.671 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_lock 2023-07-29T13:29:13.671 INFO:tasks.workunit.client.0.smithi040.stdout:test api_lock on pid 90710 2023-07-29T13:29:13.673 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.673 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.674 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.674 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.674 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.674 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90710 2023-07-29T13:29:13.674 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.674 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_lock on pid 90710' 2023-07-29T13:29:13.674 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.675 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90710 2023-07-29T13:29:13.675 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.675 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.675 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.675 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.675 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.675 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.675 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.676 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.676 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.676 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.676 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.676 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.676 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.676 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_lock_pp 2023-07-29T13:29:13.677 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.677 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.677 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.677 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_lock_pp' 2023-07-29T13:29:13.677 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_aio.log 2023-07-29T13:29:13.677 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2023-07-29T13:29:13.677 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90642/exe 2023-07-29T13:29:13.678 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.678 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.678 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.678 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.678 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.678 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.679 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.679 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.679 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.679 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.679 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.679 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.679 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.679 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.680 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.680 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.680 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.680 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.680 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.680 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.680 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.681 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.681 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.681 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.681 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.681 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.681 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.681 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_lock_pp 2023-07-29T13:29:13.681 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.682 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.682 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.682 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.682 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.682 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.682 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.682 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.682 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.683 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.683 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.683 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.683 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.683 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.683 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.683 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.684 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.684 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.684 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.684 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.685 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.685 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.685 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.686 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.686 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.686 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.686 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.687 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.687 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.687 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.687 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.688 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.688 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.688 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.689 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.689 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.689 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.690 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.690 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.690 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.690 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.691 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.691 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.691 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.691 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.692 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.692 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.692 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.692 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.692 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.693 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.693 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.693 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.694 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.694 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.694 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.694 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.694 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.695 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.695 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.695 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.695 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.695 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.695 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.695 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.695 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.696 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.696 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.696 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.696 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.696 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.696 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.696 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.697 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.697 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.697 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.697 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.697 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.697 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.697 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.697 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.698 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.698 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.698 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.698 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_aio_pp: /' 2023-07-29T13:29:13.698 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90651/exe 2023-07-29T13:29:13.699 INFO:tasks.workunit.client.0.smithi040.stdout:test api_lock_pp on pid 90736 2023-07-29T13:29:13.703 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_lock_pp 2023-07-29T13:29:13.703 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_aio_pp.log 2023-07-29T13:29:13.703 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90736 2023-07-29T13:29:13.703 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_lock_pp on pid 90736' 2023-07-29T13:29:13.703 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90736 2023-07-29T13:29:13.704 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.704 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.704 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.704 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.704 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.704 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.705 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.705 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.705 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.705 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.705 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.705 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.705 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.705 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.706 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.706 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.706 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.706 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.706 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.706 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.706 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.706 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.707 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.707 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.707 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.707 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.707 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.707 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.707 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.707 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.708 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_aio_pp 2023-07-29T13:29:13.708 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.708 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.708 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.708 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.708 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.709 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.709 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.709 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.709 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.709 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.709 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.709 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.709 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.710 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.710 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.710 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.710 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.710 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.710 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.710 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.710 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.711 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.711 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.711 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.711 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.711 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.711 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2023-07-29T13:29:13.711 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.712 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.712 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.712 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.712 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.712 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.712 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_misc 2023-07-29T13:29:13.712 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.712 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.713 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.713 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.713 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_misc' 2023-07-29T13:29:13.713 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_io.log 2023-07-29T13:29:13.713 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.713 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.714 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.714 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.714 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.714 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.714 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.714 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.715 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.715 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.715 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.715 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.715 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.715 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.715 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.716 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.716 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.716 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.716 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.716 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.716 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.716 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_io 2023-07-29T13:29:13.716 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.717 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.717 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.717 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.717 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.717 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.718 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_misc 2023-07-29T13:29:13.718 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90662/exe 2023-07-29T13:29:13.718 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_io: /' 2023-07-29T13:29:13.718 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.719 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.719 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.719 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.719 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.720 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.720 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.720 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.721 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.721 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.721 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.722 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.722 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.722 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.722 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.723 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.723 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.723 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.724 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.724 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.724 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.724 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.725 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.725 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.725 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.725 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.726 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.726 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.726 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.726 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.726 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.727 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.727 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.727 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.728 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.728 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.728 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.728 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.728 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.728 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.729 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.729 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.729 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.729 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.729 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.729 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.729 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.730 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.730 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_io_pp 2023-07-29T13:29:13.730 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.730 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.730 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.730 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.730 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.730 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.731 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.731 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.731 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.731 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.731 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.731 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.731 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.731 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.732 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.732 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.732 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.732 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.732 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.732 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90676/exe 2023-07-29T13:29:13.732 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_io_pp.log 2023-07-29T13:29:13.732 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.733 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.733 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_io_pp: /' 2023-07-29T13:29:13.733 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.733 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.733 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.733 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.733 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.734 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.734 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.734 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.734 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.734 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.734 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.734 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.734 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.735 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.735 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.735 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.735 INFO:tasks.workunit.client.0.smithi040.stdout:test api_misc on pid 90764 2023-07-29T13:29:13.736 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.736 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.737 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.737 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.737 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.737 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.737 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.737 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.737 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.737 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.738 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.738 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.738 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_misc 2023-07-29T13:29:13.738 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.738 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.738 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.738 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.739 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.739 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.739 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.739 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.739 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90764 2023-07-29T13:29:13.739 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.739 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.739 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_misc on pid 90764' 2023-07-29T13:29:13.740 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90764 2023-07-29T13:29:13.740 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.740 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.740 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.740 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.740 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.740 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.741 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_misc_pp 2023-07-29T13:29:13.741 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.741 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_misc_pp' 2023-07-29T13:29:13.741 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_asio: /' 2023-07-29T13:29:13.741 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_asio 2023-07-29T13:29:13.741 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.741 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_misc_pp 2023-07-29T13:29:13.741 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.742 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.742 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_misc_pp 2023-07-29T13:29:13.742 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90774 2023-07-29T13:29:13.742 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_misc_pp on pid 90774' 2023-07-29T13:29:13.742 INFO:tasks.workunit.client.0.smithi040.stdout:test api_misc_pp on pid 90774 2023-07-29T13:29:13.745 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90774 2023-07-29T13:29:13.745 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.745 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.745 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2023-07-29T13:29:13.745 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_tier_pp 2023-07-29T13:29:13.746 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_tier_pp' 2023-07-29T13:29:13.746 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_asio.log 2023-07-29T13:29:13.746 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_tier_pp 2023-07-29T13:29:13.746 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.746 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.746 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.746 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.746 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.747 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.747 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.747 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2023-07-29T13:29:13.747 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.747 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.747 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.747 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.748 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.748 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.748 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.748 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.748 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.748 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.748 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.748 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.749 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.749 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.749 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.749 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.749 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.749 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.749 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.749 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.750 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.750 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.750 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.750 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.750 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.750 INFO:tasks.workunit.client.0.smithi040.stdout:test api_tier_pp on pid 90784 2023-07-29T13:29:13.752 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_tier_pp 2023-07-29T13:29:13.753 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90784 2023-07-29T13:29:13.753 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_tier_pp on pid 90784' 2023-07-29T13:29:13.753 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90784 2023-07-29T13:29:13.753 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.753 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.753 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.754 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.754 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.754 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.754 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.754 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.754 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.754 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.755 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.755 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.755 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.755 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.755 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.755 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.755 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.755 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.756 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.756 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.756 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.756 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2023-07-29T13:29:13.756 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.756 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.756 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.757 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.757 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.757 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.757 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.757 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.757 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.757 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.757 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.758 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.758 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.758 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.758 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.758 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.758 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.758 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.758 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.759 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.759 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.759 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.759 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.759 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.759 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.759 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.759 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.760 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.760 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.760 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.760 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.760 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.760 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.760 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.761 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.761 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.761 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.761 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.761 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.761 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.761 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.762 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.762 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.762 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.762 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.762 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.762 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.762 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.762 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.763 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.763 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.763 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.763 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.763 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.763 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.763 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.764 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.764 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.764 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.764 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.764 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.764 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.764 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.764 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.765 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.765 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.765 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.765 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.765 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.765 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.765 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_pool 2023-07-29T13:29:13.765 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_pool' 2023-07-29T13:29:13.766 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.766 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.766 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.766 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.766 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.766 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_pool 2023-07-29T13:29:13.766 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.767 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.767 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.767 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.767 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.767 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.767 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.767 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.767 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.768 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.768 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.768 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.768 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.768 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.768 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.768 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.768 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.769 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.769 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.769 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.769 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.769 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.769 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.769 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.769 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.770 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.770 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.770 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.770 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.770 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.770 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.770 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.770 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.771 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.771 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.771 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.771 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.771 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.771 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.771 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.771 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.772 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.772 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.772 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.772 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.772 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.772 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.772 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.773 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.773 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.773 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.773 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.773 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.773 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.773 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.774 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.774 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.774 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.774 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.774 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.774 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.774 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.775 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.775 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.775 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.775 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.775 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.775 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.775 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.775 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.776 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.776 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.776 INFO:tasks.workunit.client.0.smithi040.stdout:test api_pool on pid 90808 2023-07-29T13:29:13.782 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_pool 2023-07-29T13:29:13.782 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90808 2023-07-29T13:29:13.782 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_pool on pid 90808' 2023-07-29T13:29:13.782 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90808 2023-07-29T13:29:13.782 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.782 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.782 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2023-07-29T13:29:13.783 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.783 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.783 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.783 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.783 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.783 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.783 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.784 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.784 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_snapshots 2023-07-29T13:29:13.784 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.784 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.784 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.784 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.785 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.785 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.785 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.785 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.785 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.785 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.785 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.785 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.786 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.786 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.786 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.786 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.786 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.786 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.786 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_snapshots' 2023-07-29T13:29:13.786 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.787 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.787 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.787 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.787 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.787 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.787 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.787 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.787 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90710/exe 2023-07-29T13:29:13.788 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.788 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.788 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.788 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.789 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.789 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.789 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.789 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.789 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.789 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.789 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.789 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.790 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.790 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.790 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.790 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.790 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.790 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.790 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.790 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.791 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.791 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.791 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.791 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.791 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.791 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.791 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.792 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.792 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.792 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.792 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.792 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.793 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_snapshots 2023-07-29T13:29:13.793 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.793 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.793 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.794 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.794 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.794 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.794 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.794 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.794 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.794 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.795 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.795 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.795 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.795 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.795 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.796 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.796 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.796 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.796 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.796 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.796 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.796 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.797 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.797 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.797 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.797 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.797 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.797 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.797 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.797 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.798 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.798 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.798 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.798 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.798 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.798 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.798 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.799 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.799 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.799 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.799 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.799 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.799 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.799 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.799 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.800 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.800 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.800 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.800 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.800 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.800 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.800 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.801 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.801 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.801 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.801 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.801 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.802 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.802 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.802 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.802 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90693/exe 2023-07-29T13:29:13.803 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_lock.log 2023-07-29T13:29:13.803 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_lock 2023-07-29T13:29:13.803 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_lock: /' 2023-07-29T13:29:13.803 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.804 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.804 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.804 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.804 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.805 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.805 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.805 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.805 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.806 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.806 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.806 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.806 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.807 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.807 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.807 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.807 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.808 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.808 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.808 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.809 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.809 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.809 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.809 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.810 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.810 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.810 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.810 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.811 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.811 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.811 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.811 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.812 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.812 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.812 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.812 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.813 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.813 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.813 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.813 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.814 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.814 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.814 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.814 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.815 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.815 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.815 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.815 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.816 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.816 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.816 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.816 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.817 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.817 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.817 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_snapshots 2023-07-29T13:29:13.817 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.818 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.818 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.819 INFO:tasks.workunit.client.0.smithi040.stdout:test api_snapshots on pid 90872 2023-07-29T13:29:13.826 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90872 2023-07-29T13:29:13.826 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_snapshots on pid 90872' 2023-07-29T13:29:13.827 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90872 2023-07-29T13:29:13.827 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.827 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.827 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_list 2023-07-29T13:29:13.828 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_snapshots_pp 2023-07-29T13:29:13.828 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.828 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_list.log 2023-07-29T13:29:13.829 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.829 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.829 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.829 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.830 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.830 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.830 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.830 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.831 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.831 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.831 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.831 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.832 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.832 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.832 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.832 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.833 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.833 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.833 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.833 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.834 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.834 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.834 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.834 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.835 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.835 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.835 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.836 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.836 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.836 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.836 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.837 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.837 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.837 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.837 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.838 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.838 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.838 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.838 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2023-07-29T13:29:13.839 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90784/exe 2023-07-29T13:29:13.839 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.839 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.839 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.839 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_list: /' 2023-07-29T13:29:13.839 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.840 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.840 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.840 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.840 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.840 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.840 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.840 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.841 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.841 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.841 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.841 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.841 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.841 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.841 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.841 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.842 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.842 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.842 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.842 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.842 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.842 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.842 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.842 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.843 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.843 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.843 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.843 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.843 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.843 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.844 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.844 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.844 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.844 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.844 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.844 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.845 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.845 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.845 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.845 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.845 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.846 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.846 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90764/exe 2023-07-29T13:29:13.846 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.846 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.846 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.846 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.846 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.847 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.847 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.847 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.847 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.847 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.847 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.847 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.848 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.848 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.848 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.848 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.848 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.848 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.848 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.849 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.849 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.849 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.849 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.849 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.849 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.849 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.849 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.850 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.850 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.850 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.850 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.851 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.851 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.851 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.851 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.851 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.851 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.851 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.851 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.852 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_snapshots_pp' 2023-07-29T13:29:13.852 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90736/exe 2023-07-29T13:29:13.852 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.852 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_tier_pp: /' 2023-07-29T13:29:13.852 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.852 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.852 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.853 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.853 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.853 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.853 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.853 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.853 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.853 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.853 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.854 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_snapshots_pp 2023-07-29T13:29:13.854 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.854 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.854 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.854 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.854 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.854 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.855 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.855 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_misc 2023-07-29T13:29:13.855 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.855 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.855 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.855 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.855 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.855 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.856 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.856 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.856 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.856 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.856 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.856 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.856 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.856 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.857 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.857 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.857 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.857 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_tier_pp 2023-07-29T13:29:13.858 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_tier_pp.log 2023-07-29T13:29:13.858 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.858 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.858 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.859 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.859 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.859 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.859 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.859 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.860 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.860 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.860 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.860 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.860 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.860 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.860 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.861 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.861 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.861 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.861 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.861 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.861 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.861 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.861 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_lock_pp 2023-07-29T13:29:13.862 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.862 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.862 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.862 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.862 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.862 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.862 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.862 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.863 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.863 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.863 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.863 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.863 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.863 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.863 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.863 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.864 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.864 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90774/exe 2023-07-29T13:29:13.864 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.864 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.864 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.864 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.864 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.865 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.865 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.865 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.865 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_misc.log 2023-07-29T13:29:13.865 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.865 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.865 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.865 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.866 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.866 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_lock_pp: /' 2023-07-29T13:29:13.866 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_misc_pp 2023-07-29T13:29:13.866 INFO:tasks.workunit.client.0.smithi040.stdout:test api_snapshots_pp on pid 90913 2023-07-29T13:29:13.869 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_snapshots_pp 2023-07-29T13:29:13.869 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_misc: /' 2023-07-29T13:29:13.869 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_misc_pp.log 2023-07-29T13:29:13.869 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90913 2023-07-29T13:29:13.869 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_snapshots_pp on pid 90913' 2023-07-29T13:29:13.869 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90913 2023-07-29T13:29:13.870 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.870 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.870 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_lock_pp.log 2023-07-29T13:29:13.870 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.870 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.870 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.870 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.871 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.871 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_misc_pp: /' 2023-07-29T13:29:13.871 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2023-07-29T13:29:13.871 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.871 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.871 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.871 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.871 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.872 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.872 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.872 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.872 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.872 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.872 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.872 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.872 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.873 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.873 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.873 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.873 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.873 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.873 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.873 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.873 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.874 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.874 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.874 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.874 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.874 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.874 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.874 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.874 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.875 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.875 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.875 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.875 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.875 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.875 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.875 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.875 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.876 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.876 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_stat 2023-07-29T13:29:13.876 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.876 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.876 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.876 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.876 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.877 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.877 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.877 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.877 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.877 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.877 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_stat' 2023-07-29T13:29:13.877 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.877 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.878 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.878 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.878 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.878 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.878 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.878 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.878 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.878 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.879 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.879 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.879 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.879 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.879 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.879 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.879 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.879 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.880 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.880 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.880 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.880 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.880 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.880 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.880 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.880 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.881 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90808/exe 2023-07-29T13:29:13.881 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.881 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.881 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.881 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.881 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.881 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.881 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.882 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.882 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.882 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.882 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.882 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.882 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.882 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.882 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.883 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.883 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.883 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.883 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.883 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.883 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.883 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.883 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.884 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.884 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_pool.log 2023-07-29T13:29:13.884 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.884 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.884 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.884 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.884 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.884 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.885 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.885 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.885 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.885 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.885 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.885 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.885 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.885 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.886 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.886 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.886 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.886 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.886 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.886 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_stat 2023-07-29T13:29:13.886 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_pool 2023-07-29T13:29:13.887 INFO:tasks.workunit.client.0.smithi040.stdout:test api_stat on pid 90990 2023-07-29T13:29:13.888 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_stat 2023-07-29T13:29:13.889 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_pool: /' 2023-07-29T13:29:13.889 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90990 2023-07-29T13:29:13.889 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_stat on pid 90990' 2023-07-29T13:29:13.889 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=90990 2023-07-29T13:29:13.889 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.889 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.890 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2023-07-29T13:29:13.890 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.890 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.890 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.890 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.890 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.890 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.890 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.891 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.891 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.891 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.891 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.891 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.891 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.891 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.891 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.892 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.892 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.892 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.892 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.892 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.892 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.892 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.892 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.893 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.893 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.893 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_stat_pp 2023-07-29T13:29:13.893 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_stat_pp' 2023-07-29T13:29:13.893 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_stat_pp 2023-07-29T13:29:13.893 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.893 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.893 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.894 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.894 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.894 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.894 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.894 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.894 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.894 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.894 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.895 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.895 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.895 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.895 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.895 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.895 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.895 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.896 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.896 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.896 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.896 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.896 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.896 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.896 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.896 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.897 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.897 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.897 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.897 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.897 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.897 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.897 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.897 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.898 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.898 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.898 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.898 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.898 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.898 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.898 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.898 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.899 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.899 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.899 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.899 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.899 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.899 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.899 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.899 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.900 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.900 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.900 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.900 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.900 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.900 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.900 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.900 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.901 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.901 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.901 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.901 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.901 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.901 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.901 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.902 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.902 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.902 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.902 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.902 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.902 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.902 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.902 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.903 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.903 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.903 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.903 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.903 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.903 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.903 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_stat_pp 2023-07-29T13:29:13.903 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.904 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.904 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.904 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91028 2023-07-29T13:29:13.904 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.904 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.904 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_stat_pp on pid 91028' 2023-07-29T13:29:13.905 INFO:tasks.workunit.client.0.smithi040.stdout:test api_stat_pp on pid 91028 2023-07-29T13:29:13.907 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91028 2023-07-29T13:29:13.907 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.907 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.907 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.907 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.908 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.908 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.908 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.908 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.908 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.908 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.908 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_watch_notify 2023-07-29T13:29:13.909 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.909 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_watch_notify' 2023-07-29T13:29:13.909 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.909 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.909 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2023-07-29T13:29:13.909 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.909 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.909 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.910 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.910 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.910 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.910 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.910 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.910 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.910 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90913/exe 2023-07-29T13:29:13.910 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.911 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.911 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.911 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.911 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.911 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.911 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.911 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.911 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.912 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.912 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_watch_notify 2023-07-29T13:29:13.912 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.912 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.912 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.912 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.912 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.913 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.913 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.913 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.913 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.913 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.913 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.913 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.913 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.914 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.914 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.914 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.914 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.914 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.914 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.914 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.914 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.915 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.915 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.915 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.915 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.915 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.915 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.915 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.915 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.916 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_snapshots_pp 2023-07-29T13:29:13.916 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.916 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.916 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.916 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.916 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_watch_notify 2023-07-29T13:29:13.916 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_snapshots_pp: /' 2023-07-29T13:29:13.916 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.917 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.917 INFO:tasks.workunit.client.0.smithi040.stdout:test api_watch_notify on pid 91041 2023-07-29T13:29:13.918 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.918 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.918 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.918 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91041 2023-07-29T13:29:13.918 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_watch_notify on pid 91041' 2023-07-29T13:29:13.918 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91041 2023-07-29T13:29:13.919 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.919 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.919 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2023-07-29T13:29:13.919 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.919 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.919 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.919 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.919 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.920 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.920 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.920 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.920 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.920 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.920 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.920 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.921 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.921 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.921 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.921 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.921 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.921 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.921 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.921 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.922 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.922 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.922 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.922 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [==========] Running 11 tests from 4 test suites. 2023-07-29T13:29:13.922 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [----------] Global test environment set-up. 2023-07-29T13:29:13.922 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2023-07-29T13:29:13.923 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2023-07-29T13:29:13.923 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2023-07-29T13:29:13.923 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2023-07-29T13:29:13.923 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.923 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [----------] 1 test from LibRadosMiscConnectFailure 2023-07-29T13:29:13.923 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2023-07-29T13:29:13.923 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 0 2023-07-29T13:29:13.923 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.736+0000 7f9a0b838780 10 monclient: get_monmap_and_config 2023-07-29T13:29:13.924 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.736+0000 7f9a0b838780 10 monclient: build_initial_monmap 2023-07-29T13:29:13.924 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.736+0000 7f9a0b838780 10 monclient: monmap: 2023-07-29T13:29:13.924 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: epoch 0 2023-07-29T13:29:13.924 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:29:13.924 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: last_changed 2023-07-29T13:29:13.737336+0000 2023-07-29T13:29:13.924 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: created 2023-07-29T13:29:13.737336+0000 2023-07-29T13:29:13.924 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: min_mon_release 0 (unknown) 2023-07-29T13:29:13.925 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: election_strategy: 1 2023-07-29T13:29:13.925 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T13:29:13.925 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T13:29:13.925 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T13:29:13.925 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T13:29:13.925 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T13:29:13.925 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.925 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.745+0000 7f9a0b838780 1 Processor -- start 2023-07-29T13:29:13.926 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.746+0000 7f9a0b838780 1 -- start start 2023-07-29T13:29:13.926 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.746+0000 7f9a0b838780 10 monclient: init 2023-07-29T13:29:13.926 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 10 monclient: _reopen_session rank -1 2023-07-29T13:29:13.926 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 10 monclient: _add_conns ranks=[3,2,0,1,4] 2023-07-29T13:29:13.926 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 1 --2- >> v2:172.21.15.40:3300/0 conn(0x557ec1a05a00 0x557ec1a985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:13.926 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 10 monclient(hunting): picked mon.noname-a con 0x557ec1a05a00 addr v2:172.21.15.40:3300/0 2023-07-29T13:29:13.926 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x557ec1a98ae0 0x557ec1a03760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:13.926 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 10 monclient(hunting): picked mon.noname-b con 0x557ec1a98ae0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:29:13.927 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 10 monclient(hunting): picked mon.noname-a-legacy con 0x557ec1a03ca0 addr v1:172.21.15.40:6789/0 2023-07-29T13:29:13.927 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x557ec0db63c0 con 0x557ec1a03ca0 2023-07-29T13:29:13.927 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 10 monclient(hunting): start opening mon connection 2023-07-29T13:29:13.927 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x557ec0e18d70 con 0x557ec1a05a00 2023-07-29T13:29:13.927 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 10 monclient(hunting): start opening mon connection 2023-07-29T13:29:13.927 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ec0db4300 con 0x557ec1a98ae0 2023-07-29T13:29:13.927 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.747+0000 7f9a0b838780 10 monclient(hunting): _renew_subs 2023-07-29T13:29:13.928 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90990/exe 2023-07-29T13:29:13.928 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.928 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.929 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.929 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.929 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.929 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.929 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.929 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.929 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.929 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.930 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.930 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.930 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.930 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.930 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.930 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.930 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.930 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.931 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.931 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.931 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.931 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2023-07-29T13:29:13.931 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.931 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.931 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.931 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.932 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.932 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.932 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_watch_notify_pp 2023-07-29T13:29:13.932 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.932 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.932 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.932 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.932 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.933 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.933 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.933 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.933 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.933 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.933 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.933 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.934 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.934 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.934 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.934 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.934 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.934 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.934 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.934 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.935 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.935 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.935 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.935 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.935 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.935 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.935 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.935 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.936 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.936 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.936 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.936 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.936 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.936 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.936 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.936 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.937 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.937 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_stat: /' 2023-07-29T13:29:13.937 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_watch_notify_pp' 2023-07-29T13:29:13.937 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/90872/exe 2023-07-29T13:29:13.937 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29 api_asio: [==========] Running 12 tests from 1 test suite. 2023-07-29T13:29:13.937 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [----------] Global test environment set-up. 2023-07-29T13:29:13.938 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [----------] 12 tests from AsioRados 2023-07-29T13:29:13.938 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.745+0000 7fdba6a34780 10 monclient: get_monmap_and_config 2023-07-29T13:29:13.938 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.745+0000 7fdba6a34780 10 monclient: build_initial_monmap 2023-07-29T13:29:13.938 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.745+0000 7fdba6a34780 10 monclient: monmap: 2023-07-29T13:29:13.938 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: epoch 0 2023-07-29T13:29:13.938 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:29:13.938 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: last_changed 0.000000 2023-07-29T13:29:13.939 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: created 0.000000 2023-07-29T13:29:13.939 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: min_mon_release 0 (unknown) 2023-07-29T13:29:13.939 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: election_strategy: 1 2023-07-29T13:29:13.939 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:29:13.939 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:29:13.939 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:29:13.939 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.939 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.745+0000 7fdba6a34780 1 Processor -- start 2023-07-29T13:29:13.940 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.745+0000 7fdba6a34780 1 -- start start 2023-07-29T13:29:13.940 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.745+0000 7fdba6a34780 10 monclient: init 2023-07-29T13:29:13.940 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 10 monclient: _reopen_session rank -1 2023-07-29T13:29:13.940 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:29:13.940 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x5612b5416d50 0x5612b54a6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:13.940 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 10 monclient(hunting): picked mon.noname-c con 0x5612b5416d50 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:29:13.940 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 0x5612b54a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:13.940 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 10 monclient(hunting): picked mon.noname-b con 0x5612b54ab210 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:29:13.941 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5612b54ae710 0x5612b54ac460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:13.941 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 10 monclient(hunting): picked mon.noname-a con 0x5612b54ae710 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:29:13.941 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 10 monclient(hunting): start opening mon connection 2023-07-29T13:29:13.941 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b5395d70 con 0x5612b54ae710 2023-07-29T13:29:13.941 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 10 monclient(hunting): start opening mon connection 2023-07-29T13:29:13.941 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b5331300 con 0x5612b54ab210 2023-07-29T13:29:13.941 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 10 monclient(hunting): start opening mon connection 2023-07-29T13:29:13.941 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612b5330fa0 con 0x5612b5416d50 2023-07-29T13:29:13.942 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 10 monclient(hunting): _renew_subs 2023-07-29T13:29:13.942 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.746+0000 7fdba6a34780 10 monclient(hunting): authenticate will time out at 1475.879716s 2023-07-29T13:29:13.943 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_stat 2023-07-29T13:29:13.943 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_stat.log 2023-07-29T13:29:13.944 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.944 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_watch_notify_pp 2023-07-29T13:29:13.944 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.944 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.944 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.944 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.944 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.945 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.945 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.945 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.945 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.945 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.945 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.945 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.945 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.946 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.946 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.946 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.946 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.946 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.946 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.946 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.946 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.947 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.947 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.947 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.947 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.947 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.947 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.947 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.947 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.948 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.948 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.948 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.948 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.948 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.948 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.948 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.948 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.949 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.949 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.949 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_watch_notify_pp 2023-07-29T13:29:13.949 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_snapshots 2023-07-29T13:29:13.949 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91080 2023-07-29T13:29:13.949 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-test api_watch_notify_pp on pid 91080 2023-07-29T13:29:13.951 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_watch_notify_pp on pid 91080' 2023-07-29T13:29:13.951 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91080 2023-07-29T13:29:13.951 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.951 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.951 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2023-07-29T13:29:13.951 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.952 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.952 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.952 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.952 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.952 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_cmd 2023-07-29T13:29:13.952 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.952 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_cmd' 2023-07-29T13:29:13.952 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_snapshots.log 2023-07-29T13:29:13.953 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.953 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_cmd 2023-07-29T13:29:13.953 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_snapshots: /' 2023-07-29T13:29:13.953 INFO:tasks.workunit.client.0.smithi040.stdout:test api_cmd on pid 91092 2023-07-29T13:29:13.954 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_cmd 2023-07-29T13:29:13.954 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91092 2023-07-29T13:29:13.954 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_cmd on pid 91092' 2023-07-29T13:29:13.954 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91092 2023-07-29T13:29:13.955 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.955 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.955 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_cmd_pp 2023-07-29T13:29:13.955 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_cmd_pp' 2023-07-29T13:29:13.955 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_cmd_pp 2023-07-29T13:29:13.955 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.955 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.956 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.956 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.956 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.956 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.956 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.956 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.956 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.956 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.957 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.957 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.957 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.957 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.957 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.957 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.957 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.958 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.958 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.958 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.958 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.958 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.958 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.958 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.958 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.959 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.959 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.959 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.959 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.959 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.959 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.959 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.959 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.960 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.960 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.960 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.960 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.960 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.960 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.961 INFO:tasks.workunit.client.0.smithi040.stdout:test api_cmd_pp on pid 91104 2023-07-29T13:29:13.963 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_cmd_pp 2023-07-29T13:29:13.963 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91104 2023-07-29T13:29:13.963 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_cmd_pp on pid 91104' 2023-07-29T13:29:13.963 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91104 2023-07-29T13:29:13.963 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.964 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.964 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_service 2023-07-29T13:29:13.964 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2023-07-29T13:29:13.964 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_service' 2023-07-29T13:29:13.964 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.964 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.964 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.965 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.965 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.965 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_service 2023-07-29T13:29:13.965 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.965 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.965 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.965 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.965 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.966 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.966 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.966 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.966 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.966 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.966 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.966 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.966 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.967 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.967 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.967 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.967 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.967 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.967 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.967 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.967 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.968 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.968 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.968 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.968 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.968 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.968 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.968 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.969 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.969 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.969 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.969 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.969 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.969 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.969 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.969 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.970 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.970 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.970 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.970 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.970 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.970 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.970 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.970 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.971 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.971 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.971 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.971 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.971 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.971 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.971 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.971 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.972 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.972 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.972 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.972 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.972 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91028/exe 2023-07-29T13:29:13.972 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.972 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:13.972 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:13.973 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:13.973 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.973 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:13.973 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.973 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:13.973 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:13.973 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.973 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:13.974 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.974 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:13.974 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.974 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:13.974 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.974 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:13.974 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:13.974 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.975 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.975 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.975 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:13.975 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.975 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.975 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.975 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.975 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.976 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.976 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.976 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.976 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.976 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.976 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:13.976 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.977 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.977 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.977 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.977 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.977 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.977 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.977 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.977 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.978 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.978 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.978 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.978 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.978 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.978 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.978 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.978 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.979 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.979 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.979 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.979 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.979 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.979 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.979 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.979 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.980 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.980 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.980 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.980 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2023-07-29T13:29:13.980 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.980 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.980 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.980 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.981 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.981 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.981 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.981 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.981 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_stat_pp.log 2023-07-29T13:29:13.981 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_stat_pp: /' 2023-07-29T13:29:13.981 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.981 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_service 2023-07-29T13:29:13.982 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.982 INFO:tasks.workunit.client.0.smithi040.stdout:test api_service on pid 91128 2023-07-29T13:29:13.983 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91128 2023-07-29T13:29:13.984 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_service on pid 91128' 2023-07-29T13:29:13.984 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91128 2023-07-29T13:29:13.984 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:13.984 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:13.984 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.984 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.985 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.985 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.985 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_stat_pp 2023-07-29T13:29:13.985 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2023-07-29T13:29:13.985 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.985 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:13.985 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.985 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:13.986 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:13.986 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:13.986 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:13.986 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:13.986 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:13.986 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.986 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:13.986 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:13.987 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:13.987 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:13.987 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:13.987 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:13.987 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:13.987 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.987 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:13.987 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:13.988 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:13.988 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:13.988 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:13.988 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:13.988 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:13.988 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:13.988 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.989 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:13.989 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.989 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.989 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:13.989 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:13.989 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.989 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:13.989 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:13.990 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.990 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:13.990 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:13.990 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.990 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:13.990 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:13.990 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.990 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:13.991 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.991 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:13.991 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:13.991 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:13.991 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:13.991 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.991 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:13.991 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.992 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:13.992 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.992 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_service_pp 2023-07-29T13:29:13.992 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_service_pp' 2023-07-29T13:29:13.992 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.992 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.992 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.992 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.993 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.993 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.993 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.993 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.993 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.993 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.993 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.993 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.994 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.994 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.994 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.994 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.994 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.994 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:13.994 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.994 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91041/exe 2023-07-29T13:29:13.995 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:13.995 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:13.995 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:13.995 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:13.995 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.995 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:13.995 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.995 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:13.996 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:13.996 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:13.996 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:13.996 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:13.996 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:13.996 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:13.996 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:13.997 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:13.997 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:13.997 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:13.997 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:13.997 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:13.997 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:13.997 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:13.997 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:13.998 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_watch_notify: /' 2023-07-29T13:29:13.998 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.998 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:13.998 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.998 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:13.998 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:13.998 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.998 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:13.999 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.999 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:13.999 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.999 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:13.999 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:13.999 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:13.999 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:13.999 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:14.000 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.000 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_watch_notify.log 2023-07-29T13:29:14.000 INFO:tasks.workunit.client.0.smithi040.stdout:29T13:29:13.752+0000 7fdba35fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 0x5612b54a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:29:14.000 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.756+0000 7fdba35fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 0x5612b54a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:48794/0 (socket says 172.21.15.40:48794) 2023-07-29T13:29:14.000 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.756+0000 7fdba35fc640 1 -- 172.21.15.40:0/948898658 learned_addr learned my addr 172.21.15.40:0/948898658 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:29:14.001 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.756+0000 7fdba35fc640 10 monclient(hunting): get_auth_request con 0x5612b54ab210 auth_method 0 2023-07-29T13:29:14.001 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.756+0000 7fdba35fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:29:14.001 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.756+0000 7fdba35fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:29:14.001 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.756+0000 7fdba35fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:29:14.001 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.765+0000 7fdba45fe640 1 --2- 172.21.15.40:0/948898658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5612b54ae710 0x5612b54ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:29:14.001 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.779+0000 7fdba35fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:29:14.001 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.779+0000 7fdba35fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:29:14.001 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.779+0000 7fdba35fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:29:14.002 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.780+0000 7fdba3dfd640 1 --2- 172.21.15.40:0/948898658 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x5612b5416d50 0x5612b54a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:29:14.002 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.780+0000 7fdba3dfd640 10 monclient(hunting): get_auth_request con 0x5612b5416d50 auth_method 0 2023-07-29T13:29:14.002 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.780+0000 7fdba3dfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:29:14.002 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.780+0000 7fdba3dfd640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:29:14.002 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.780+0000 7fdba3dfd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:29:14.002 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.782+0000 7fdba3dfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:29:14.002 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.782+0000 7fdba3dfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:29:14.002 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.782+0000 7fdba3dfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:29:14.003 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.789+0000 7fdba35fc640 10 monclient(hunting): handle_auth_done global_id 4390 payload 274 2023-07-29T13:29:14.003 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.789+0000 7fdba35fc640 1 -- 172.21.15.40:0/948898658 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x5612b5416d50 msgr2=0x5612b54a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:29:14.003 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.789+0000 7fdba35fc640 1 --2- 172.21.15.40:0/948898658 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x5612b5416d50 0x5612b54a6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.004 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:14.005 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:14.005 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:14.005 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.005 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:14.005 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.005 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:14.005 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:14.006 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.006 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:14.006 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.006 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:14.006 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.006 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:14.006 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.006 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:14.007 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:14.007 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_watch_notify 2023-07-29T13:29:14.007 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:14.007 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:14.007 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:14.007 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.007 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:14.007 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.008 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:14.008 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:14.008 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:14.008 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.008 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:14.008 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.008 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:14.009 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:14.009 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:14.009 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.009 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:14.009 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.009 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:14.009 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.009 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:14.010 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:14.010 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:14.010 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.010 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:14.010 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.010 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:14.010 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:14.010 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.011 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_service_pp 2023-07-29T13:29:14.011 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:14.011 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:14.011 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:14.011 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:14.011 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.011 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:14.011 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.012 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:14.012 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:14.012 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.012 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:14.012 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:14.012 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:14.012 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:14.012 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.013 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:14.013 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.013 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:14.013 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:14.013 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:14.013 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.013 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:14.013 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.014 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:14.014 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:14.014 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:14.014 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.014 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:14.014 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.014 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:14.014 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.015 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:14.015 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:14.015 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:14.015 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.015 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:14.015 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.015 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:14.015 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:14.016 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.016 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:14.016 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.016 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:14.016 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.016 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.016 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:14.017 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:14.017 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.017 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:14.017 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.017 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:14.017 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:14.017 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:14.018 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.789+0000 7fdba35fc640 1 -- 172.21.15.40:0/948898658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6test api_service_pp on pid 91183 2023-07-29T13:29:14.020 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_service_pp 2023-07-29T13:29:14.020 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91183 2023-07-29T13:29:14.020 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_service_pp on pid 91183' 2023-07-29T13:29:14.020 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91183 2023-07-29T13:29:14.020 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:14.020 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:14.020 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_c_write_operations 2023-07-29T13:29:14.021 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2023-07-29T13:29:14.021 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_c_write_operations' 2023-07-29T13:29:14.021 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:14.021 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:14.021 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:14.021 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:14.021 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:14.022 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:14.022 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.022 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:14.022 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.022 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:14.022 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:14.022 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.022 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:14.023 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:14.023 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:14.023 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:14.023 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.023 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:14.023 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.023 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:14.023 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:14.024 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.024 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.024 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:14.024 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.024 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_c_write_operations 2023-07-29T13:29:14.024 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:14.024 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.024 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:14.025 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:14.025 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.025 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:14.025 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.025 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:14.025 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:14.025 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:14.025 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:14.026 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:14.026 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:14.026 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:14.026 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:14.026 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.026 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:14.026 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.027 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:14.027 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:14.027 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.027 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:14.027 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.027 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:14.027 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:14.027 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:14.028 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.028 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.028 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:14.028 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:14.028 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.028 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.028 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:14.028 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:14.029 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:14.029 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:14.029 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.029 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:14.029 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.029 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:14.029 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.029 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.030 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.030 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:14.030 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.030 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:14.030 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.030 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:14.030 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:14.030 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:14.031 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.031 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:14.031 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.031 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:14.031 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:14.031 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.031 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:14.032 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:14.032 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:14.032 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:14.032 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.032 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:14.032 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.032 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:14.032 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91080/exe 2023-07-29T13:29:14.033 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:14.033 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.033 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.033 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:14.033 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.033 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:14.033 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.033 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:14.034 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:14.034 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.034 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:14.034 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.034 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:14.034 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:14.034 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:14.034 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.035 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:14.035 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.035 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:14.035 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:14.035 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91104/exe 2023-07-29T13:29:14.035 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:14.035 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:14.035 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:14.036 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:14.036 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:14.036 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:14.036 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:14.036 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:14.036 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:14.036 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:14.036 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:14.037 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:14.037 INFO:tasks.workunit.client.0.smithi040.stdout:test api_c_write_operations on pid 91214 2023-07-29T13:29:14.040 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_c_write_operations 2023-07-29T13:29:14.040 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:14.040 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91214 2023-07-29T13:29:14.040 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:14.040 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_c_write_operations on pid 91214' 2023-07-29T13:29:14.040 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91214 2023-07-29T13:29:14.041 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:14.041 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:14.041 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:14.041 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:14.041 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.041 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:14.041 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:14.042 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:14.042 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.042 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:14.042 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.042 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:14.042 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:14.042 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:14.042 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.043 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.043 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:14.043 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:14.043 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.043 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:14.043 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:14.043 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_watch_notify_pp 2023-07-29T13:29:14.043 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.044 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:14.044 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:14.044 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.044 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:14.044 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:14.044 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:14.044 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.044 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:14.045 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.045 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:14.045 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:14.045 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:14.045 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.045 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:14.045 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:14.045 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:14.046 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:14.046 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.046 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.046 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:14.046 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:14.046 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.046 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:14.046 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.047 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:14.047 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:14.047 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:14.047 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:14.047 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:14.047 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2023-07-29T13:29:14.047 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2023-07-29T13:29:14.047 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:14.048 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:14.048 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:14.048 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_cmd_pp 2023-07-29T13:29:14.048 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.048 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:14.048 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.048 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:14.049 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:14.049 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.049 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:14.049 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.049 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:14.049 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.049 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:14.049 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.050 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:14.050 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:14.050 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2023-07-29T13:29:14.050 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91128/exe 2023-07-29T13:29:14.050 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_cmd_pp: /' 2023-07-29T13:29:14.050 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_c_read_operations 2023-07-29T13:29:14.050 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_c_read_operations' 2023-07-29T13:29:14.050 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2023-07-29T13:29:14.051 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:14.051 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:14.051 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:14.051 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:14.051 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_c_read_operations 2023-07-29T13:29:14.051 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:14.051 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:14.051 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:14.052 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:14.052 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:14.052 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:14.052 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:14.052 INFO:tasks.workunit.client.0.smithi040.stdout:789/0] conn(0x5612b54ae710 msgr2=0x5612b54ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:29:14.052 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.789+0000 7fdba35fc640 1 --2- 172.21.15.40:0/948898658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5612b54ae710 0x5612b54ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.789+0000 7fdba35fc640 10 monclient: _finish_hunting 0 2023-07-29T13:29:14.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.789+0000 7fdba35fc640 1 monclient: found mon.noname-b 2023-07-29T13:29:14.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.789+0000 7fdba35fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:29:14.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.789+0000 7fdba35fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:29:14.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.789+0000 7fdba35fc640 1 -- 172.21.15.40:0/948898658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94005a20 con 0x5612b54ab210 2023-07-29T13:29:14.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.792+0000 7fdba35fc640 1 --2- 172.21.15.40:0/948898658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 0x5612b54a75c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800a420 tx=0x7fdb9800a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=434c399cf5854a03 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:29:14.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 1 -- 172.21.15.40:0/948898658 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9800c8f0 con 0x5612b54ab210 2023-07-29T13:29:14.054 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:29:14.054 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:29:14.054 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 10 monclient: dump: 2023-07-29T13:29:14.054 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: epoch 1 2023-07-29T13:29:14.054 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:29:14.054 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.054 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.054 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: min_mon_release 18 (reef) 2023-07-29T13:29:14.055 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: election_strategy: 1 2023-07-29T13:29:14.055 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:29:14.055 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:29:14.055 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:29:14.055 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:14.055 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 10 monclient: _finish_auth 0 2023-07-29T13:29:14.055 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 10 monclient: _check_auth_tickets 2023-07-29T13:29:14.055 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:14.056 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba6a34780 5 monclient: authenticate success, global_id 4390 2023-07-29T13:29:14.056 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba6a34780 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T13:29:14.056 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 1 -- 172.21.15.40:0/948898658 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9800ca50 con 0x5612b54ab210 2023-07-29T13:29:14.056 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:29:14.056 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 1 -- 172.21.15.40:0/948898658 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9800ce00 con 0x5612b54ab210 2023-07-29T13:29:14.056 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:29:14.056 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:29:14.057 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba25fa640 10 monclient: dump: 2023-07-29T13:29:14.057 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: epoch 1 2023-07-29T13:29:14.057 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:29:14.057 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.057 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.057 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: min_mon_release 18 (reef) 2023-07-29T13:29:14.057 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: election_strategy: 1 2023-07-29T13:29:14.057 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:29:14.058 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:29:14.058 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:29:14.058 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:14.058 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba6a34780 10 monclient: get_monmap_and_config success 2023-07-29T13:29:14.058 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.819+0000 7fdba6a34780 4 set_mon_vals no callback set 2023-07-29T13:29:14.058 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 10 monclient: shutdown 2023-07-29T13:29:14.058 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:29:14.058 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 1 -- 172.21.15.40:0/948898658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 msgr2=0x5612b54a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:29:14.059 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 1 --2- 172.21.15.40:0/948898658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 0x5612b54a75c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800a420 tx=0x7fdb9800a8e0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.059 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 1 -- 172.21.15.40:0/948898658 shutdown_connections 2023-07-29T13:29:14.059 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 1 --2- 172.21.15.40:0/948898658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 0x5612b54a75c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.059 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 1 --2- 172.21.15.40:0/948898658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5612b54ae710 0x5612b54ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.059 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 1 --2- 172.21.15.40:0/948898658 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x5612b5416d50 0x5612b54a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.059 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 1 -- 172.21.15.40:0/948898658 >> 172.21.15.40:0/948898658 conn(0x5612b53db9b0 msgr2=0x5612b54a6280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:29:14.059 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 1 -- 172.21.15.40:0/948898658 shutdown_connections 2023-07-29T13:29:14.059 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 1 -- 172.21.15.40:0/948898658 wait complete. 2023-07-29T13:29:14.060 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 10 monclient: build_initial_monmap 2023-07-29T13:29:14.060 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.820+0000 7fdba6a34780 10 monclient: monmap: 2023-07-29T13:29:14.060 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: epoch 0 2023-07-29T13:29:14.060 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:29:14.060 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: last_changed 0.000000 2023-07-29T13:29:14.060 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: created 0.000000 2023-07-29T13:29:14.060 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: min_mon_release 0 (unknown) 2023-07-29T13:29:14.061 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: election_strategy: 1 2023-07-29T13:29:14.061 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:29:14.061 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:29:14.061 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:29:14.061 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:14.061 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.821+0000 7fdba6a34780 1 librados: starting msgr at 2023-07-29T13:29:14.061 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.821+0000 7fdba6a34780 1 librados: starting objecter 2023-07-29T13:29:14.061 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.821+0000 7fdba6a34780 1 Processor -- start 2023-07-29T13:29:14.062 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:14.062 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:14.062 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:14.063 INFO:tasks.workunit.client.0.smithi040.stdout: test api_c_read_operations on pid 91245 2023-07-29T13:29:14.064 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_c_read_operations 2023-07-29T13:29:14.065 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:14.065 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:14.065 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.065 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:14.065 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91245 2023-07-29T13:29:14.065 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_c_read_operations on pid 91245' 2023-07-29T13:29:14.065 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91245 2023-07-29T13:29:14.066 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:14.066 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:14.066 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s api_cls_remote_reads 2023-07-29T13:29:14.066 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' api_cls_remote_reads' 2023-07-29T13:29:14.066 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo api_cls_remote_reads 2023-07-29T13:29:14.066 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:14.066 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:14.066 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:14.067 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:14.067 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.067 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:14.067 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.067 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:14.067 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:14.067 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.067 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.068 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:14.068 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.068 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:14.068 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:14.068 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2023-07-29T13:29:14.068 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:14.068 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:14.068 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_service 2023-07-29T13:29:14.069 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:14.069 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:14.069 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:14.069 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_service.log 2023-07-29T13:29:14.069 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.069 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:14.069 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:14.070 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:14.070 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_service: /' 2023-07-29T13:29:14.070 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.070 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:14.070 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.070 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:14.070 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:14.070 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.071 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:14.071 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.071 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:14.071 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.071 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.071 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.071 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:14.071 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.072 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:14.072 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.072 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:14.072 INFO:tasks.workunit.client.0.smithi040.stdout:test api_cls_remote_reads on pid 91268 2023-07-29T13:29:14.073 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=api_cls_remote_reads 2023-07-29T13:29:14.073 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91268 2023-07-29T13:29:14.073 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test api_cls_remote_reads on pid 91268' 2023-07-29T13:29:14.073 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91268 2023-07-29T13:29:14.073 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:14.073 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:14.074 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.821+0000 7fdba6a34780 1 -- start start 2023-07-29T13:29:14.074 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.821+0000 7fdba6a34780 1 librados: setting wanted keys 2023-07-29T13:29:14.074 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.821+0000 7fdba6a34780 1 librados: calling monclient init 2023-07-29T13:29:14.074 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.821+0000 7fdba6a34780 10 monclient: init 2023-07-29T13:29:14.074 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 10 monclient: _reopen_session rank -1 2023-07-29T13:29:14.074 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T13:29:14.074 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x5612b5416d50 0x5612b554e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:14.075 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 10 monclient(hunting): picked mon.noname-c con 0x5612b5416d50 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:29:14.075 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 0x5612b54a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:14.075 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 10 monclient(hunting): picked mon.noname-b con 0x5612b54ab210 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:29:14.075 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5612b54ae710 0x5612b54a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:14.075 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 10 monclient(hunting): picked mon.noname-a con 0x5612b54ae710 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:29:14.075 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 10 monclient(hunting): start opening mon connection 2023-07-29T13:29:14.075 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b53143f0 con 0x5612b54ae710 2023-07-29T13:29:14.075 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 10 monclient(hunting): start opening mon connection 2023-07-29T13:29:14.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b54b9860 con 0x5612b54ab210 2023-07-29T13:29:14.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 10 monclient(hunting): start opening mon connection 2023-07-29T13:29:14.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612b54ab980 con 0x5612b5416d50 2023-07-29T13:29:14.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 10 monclient(hunting): _renew_subs 2023-07-29T13:29:14.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.822+0000 7fdba6a34780 10 monclient(hunting): authenticate will time out at 1475.955618s 2023-07-29T13:29:14.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.828+0000 7fdba35fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 0x5612b54a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:29:14.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.835+0000 7fdba35fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 0x5612b54a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:48838/0 (socket says 172.21.15.40:48838) 2023-07-29T13:29:14.077 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.835+0000 7fdba35fc640 1 -- 172.21.15.40:0/3644954068 learned_addr learned my addr 172.21.15.40:0/3644954068 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:29:14.077 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.835+0000 7fdba35fc640 10 monclient(hunting): get_auth_request con 0x5612b54ab210 auth_method 0 2023-07-29T13:29:14.079 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2023-07-29T13:29:14.080 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:14.080 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:14.080 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:14.080 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.080 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:14.080 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.081 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:14.081 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:14.081 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.081 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:14.081 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.081 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:14.081 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.081 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:14.082 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.082 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:14.082 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:14.082 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:14.082 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:14.082 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:14.082 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:14.083 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:14.083 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.083 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:14.083 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:14.083 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:14.083 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:14.083 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.083 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:14.084 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.084 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:14.084 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:14.084 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.084 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:14.084 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.084 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:14.084 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.085 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.085 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.085 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:14.085 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.085 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:14.085 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.085 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:14.085 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:14.086 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:14.086 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:14.086 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.086 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:14.086 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.086 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:14.086 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:14.086 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.087 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.087 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:14.087 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.087 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:14.087 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.087 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:14.087 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:14.087 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.088 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:14.088 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.088 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:14.088 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:14.088 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s list_parallel 2023-07-29T13:29:14.088 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' list_parallel' 2023-07-29T13:29:14.088 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.089 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:14.089 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.089 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:14.089 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:14.089 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91092/exe 2023-07-29T13:29:14.089 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:14.089 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:14.089 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:14.090 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:14.090 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:14.090 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:14.090 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:14.090 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:14.090 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:14.090 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:14.090 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:14.091 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_cmd.log 2023-07-29T13:29:14.091 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:14.091 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:14.091 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:14.091 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.091 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:14.091 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.091 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:14.092 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:14.092 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:14.092 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.092 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:14.092 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.092 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:14.092 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:14.092 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:14.093 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.093 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:14.093 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.093 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:14.093 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.093 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:14.093 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo list_parallel 2023-07-29T13:29:14.093 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:14.094 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:14.094 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.094 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:14.094 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.094 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:14.094 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:14.094 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:14.094 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.095 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:14.095 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.095 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:14.095 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.095 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.095 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.095 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:14.095 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.096 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:14.096 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.096 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:14.096 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:14.096 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:14.096 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.096 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:14.097 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.097 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:14.097 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:14.097 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:14.097 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.097 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:14.097 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.097 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:14.098 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:14.098 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:14.098 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.098 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:14.098 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.098 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:14.098 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.098 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_cmd: /' 2023-07-29T13:29:14.099 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13test list_parallel on pid 91296 2023-07-29T13:29:14.101 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=list_parallel 2023-07-29T13:29:14.101 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_cmd 2023-07-29T13:29:14.101 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91296 2023-07-29T13:29:14.101 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test list_parallel on pid 91296' 2023-07-29T13:29:14.101 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91296 2023-07-29T13:29:14.102 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:14.102 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:14.102 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s open_pools_parallel 2023-07-29T13:29:14.102 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' open_pools_parallel' 2023-07-29T13:29:14.102 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2023-07-29T13:29:14.102 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:14.102 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:14.102 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:14.103 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:14.103 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.103 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:14.103 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:14.103 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:14.103 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:14.103 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.103 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:14.104 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.104 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:14.104 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:14.104 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.104 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:14.104 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.104 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:14.105 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.105 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.105 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.105 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:14.105 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.105 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:14.105 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.105 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:14.106 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:14.106 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:14.106 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:14.106 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.106 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:14.106 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.106 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:14.106 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:14.107 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.107 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.107 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:14.107 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.107 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:14.107 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.107 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:14.107 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:14.108 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.108 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:14.108 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.108 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:14.108 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:14.108 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:14.108 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:14.109 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:14.109 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.109 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:14.109 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.109 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:14.109 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:14.109 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.109 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:14.110 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:14.110 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:14.110 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.110 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:14.110 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.110 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:14.110 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:14.110 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:14.111 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.111 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:14.111 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.111 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:14.111 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:14.111 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:14.111 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.111 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:14.112 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.112 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:14.112 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.112 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:14.112 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:14.112 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:14.112 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.112 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:14.113 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.113 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:14.113 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:14.113 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.113 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo open_pools_parallel 2023-07-29T13:29:14.113 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:14.113 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:14.113 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:14.114 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.114 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:14.114 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.114 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:14.114 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:14.114 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.114 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:14.114 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:14.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:14.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:14.115 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:14.115 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:14.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:14.116 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91183/exe 2023-07-29T13:29:14.116 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:14.116 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:14.116 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:14.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:14.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:14.117 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:14.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:14.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:14.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:14.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:14.118 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:14.118 INFO:tasks.workunit.client.0.smithi040.stdout:test open_pools_parallel on pid 91367 2023-07-29T13:29:14.119 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=open_pools_parallel 2023-07-29T13:29:14.120 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91367 2023-07-29T13:29:14.120 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:14.120 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test open_pools_parallel on pid 91367' 2023-07-29T13:29:14.120 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91367 2023-07-29T13:29:14.120 INFO:tasks.workunit.client.0.smithi040.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-07-29T13:29:14.120 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:14.120 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2023-07-29T13:29:14.121 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:14.121 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:14.121 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:14.121 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:14.121 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.121 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:14.121 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:14.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:14.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:14.122 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.122 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:14.122 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:14.122 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:14.122 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:14.122 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:14.122 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:14.123 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:14.123 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:14.123 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:14.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:14.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:14.123 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:14.124 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.124 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_service_pp 2023-07-29T13:29:14.124 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.124 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:14.124 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.124 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:14.124 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:14.124 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:14.125 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_service_pp.log 2023-07-29T13:29:14.125 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_service_pp: /' 2023-07-29T13:29:14.125 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:14.125 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:14.125 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:14.125 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:14.125 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:14.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:14.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:14.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:14.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:14.126 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:14.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.126 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:14.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:14.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:14.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:14.127 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:14.127 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:14.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:14.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:14.128 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.128 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:14.128 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:14.128 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:14.129 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.129 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:14.129 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.129 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:14.129 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:14.129 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.129 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.129 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:14.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:14.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:14.130 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:14.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:14.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:14.131 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:14.131 INFO:tasks.workunit.client.0.smithi040.stdout::29:13.835+0000 7fdba35fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:29:14.131 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.835+0000 7fdba35fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:29:14.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.835+0000 7fdba35fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:29:14.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.840+0000 7fdba35fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:29:14.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.840+0000 7fdba35fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:29:14.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.840+0000 7fdba35fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:29:14.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.852+0000 7fdba35fc640 10 monclient(hunting): handle_auth_done global_id 4402 payload 824 2023-07-29T13:29:14.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.852+0000 7fdba35fc640 1 -- 172.21.15.40:0/3644954068 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x5612b5416d50 msgr2=0x5612b554e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:29:14.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.852+0000 7fdba35fc640 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x5612b5416d50 0x5612b554e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.852+0000 7fdba35fc640 1 -- 172.21.15.40:0/3644954068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5612b54ae710 msgr2=0x5612b54a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:29:14.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.852+0000 7fdba35fc640 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5612b54ae710 0x5612b54a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.852+0000 7fdba35fc640 10 monclient: _finish_hunting 0 2023-07-29T13:29:14.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.852+0000 7fdba35fc640 1 monclient: found mon.noname-b 2023-07-29T13:29:14.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.852+0000 7fdba35fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:29:14.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.852+0000 7fdba35fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:29:14.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.852+0000 7fdba35fc640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb98005ca0 con 0x5612b54ab210 2023-07-29T13:29:14.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.863+0000 7fdba35fc640 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 0x5612b54a4440 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800ade0 tx=0x7fdb98002f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:29:14.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.863+0000 7fdba15f8640 10 client.?.objecter ms_handle_connect 0x5612b54ab210 2023-07-29T13:29:14.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.863+0000 7fdba15f8640 10 client.?.objecter resend_mon_ops 2023-07-29T13:29:14.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98015cc0 con 0x5612b54ab210 2023-07-29T13:29:14.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:29:14.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:29:14.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 10 monclient: dump: 2023-07-29T13:29:14.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: epoch 1 2023-07-29T13:29:14.135 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:29:14.135 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.135 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.135 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: min_mon_release 18 (reef) 2023-07-29T13:29:14.139 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:14.139 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:14.139 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:14.139 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:14.139 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.139 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:14.139 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.140 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:14.140 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:14.140 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.140 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:14.140 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:14.140 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.140 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:14.141 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:14.141 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.141 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:14.141 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.141 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.141 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:14.141 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:14.141 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.142 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:14.142 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:14.142 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:14.142 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.142 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:14.142 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.142 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:14.142 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.143 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.143 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.143 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:14.143 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.143 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:14.143 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.143 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:14.143 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.144 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:14.144 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.144 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:14.144 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:14.144 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:14.144 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:14.144 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:14.144 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:14.145 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.145 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:14.145 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.145 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:14.145 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:14.145 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.145 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:14.145 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:14.146 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:14.146 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:14.146 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.146 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:14.146 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.146 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:14.146 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:14.147 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:14.147 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:14.147 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:14.147 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:14.147 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.147 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:14.147 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.147 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.148 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:14.148 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:14.148 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:14.148 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.148 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:14.148 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:14.148 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.148 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:14.149 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:14.149 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.149 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:14.149 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.149 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.149 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.149 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:14.149 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.150 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:14.150 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.150 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:14.150 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:14.150 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.150 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:14.150 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.150 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:14.151 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:14.151 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91245/exe 2023-07-29T13:29:14.151 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:14.151 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:14.151 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:14.151 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:14.151 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:14.151 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:14.152 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:14.152 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:14.152 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:14.152 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:14.152 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:14.152 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:14.152 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:14.152 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:14.153 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.153 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:14.153 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.153 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:14.153 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:14.153 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.153 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:14.154 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.154 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:14.154 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_c_read_operations 2023-07-29T13:29:14.154 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.154 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:14.154 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.154 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:14.154 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:14.155 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_c_read_operations: /' 2023-07-29T13:29:14.155 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:14.155 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.155 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:14.155 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.155 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:14.155 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2023-07-29T13:29:14.155 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:14.156 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:14.156 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:14.156 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:14.156 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.156 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:14.156 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.156 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:14.156 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:14.157 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.157 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.157 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:14.157 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.157 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:14.157 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.157 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:14.157 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:14.158 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.158 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:14.158 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.158 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:14.158 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:14.158 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:14.158 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:14.158 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:14.159 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:14.159 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.159 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:14.159 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.159 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:14.159 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:14.159 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.159 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.160 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:14.160 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.160 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:14.160 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.160 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:14.160 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:14.160 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.161 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:14.161 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.161 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:14.161 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:14.161 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91268/exe 2023-07-29T13:29:14.161 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:14.161 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:14.161 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:14.162 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.162 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:14.162 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.162 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:14.162 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:14.162 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.162 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:14.162 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91296/exe 2023-07-29T13:29:14.163 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:14.163 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:14.163 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:14.163 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:14.163 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:14.163 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:14.163 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:14.163 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:14.164 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:14.164 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:14.164 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:14.164 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2023-07-29T13:29:14.164 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:14.164 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:14.164 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:14.164 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:14.165 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:14.165 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:14.165 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_cls_remote_reads 2023-07-29T13:29:14.165 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:14.165 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:14.165 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:14.165 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:14.165 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2023-07-29T13:29:14.166 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:14.166 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:14.166 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:14.166 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.166 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:14.166 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.166 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:14.166 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:14.167 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.167 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:14.167 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.167 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:14.167 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.167 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:14.167 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.167 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:14.168 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:14.168 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:14.168 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %25s delete_pools_parallel 2023-07-29T13:29:14.168 INFO:tasks.workunit.client.0.smithi040.stderr:+ r=' delete_pools_parallel' 2023-07-29T13:29:14.168 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91214/exe 2023-07-29T13:29:14.168 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.168 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:14.169 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.169 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:14.169 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:14.169 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:14.169 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:14.169 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_list_parallel.log 2023-07-29T13:29:14.169 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ list_parallel: /' 2023-07-29T13:29:14.169 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:14.170 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:14.170 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:14.170 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:14.170 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:14.170 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:14.170 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:14.170 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:14.170 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:14.171 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:14.171 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_api_c_write_operations 2023-07-29T13:29:14.171 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_list_parallel 2023-07-29T13:29:14.171 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:14.171 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:14.171 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:14.171 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.171 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:14.172 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.172 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:14.172 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:14.172 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.172 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.172 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:14.172 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.172 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:14.173 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.173 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:14.173 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:14.173 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.173 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:14.173 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.173 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:14.173 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:14.174 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2023-07-29T13:29:14.174 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ api_c_write_operations: /' 2023-07-29T13:29:14.174 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo delete_pools_parallel 2023-07-29T13:29:14.174 INFO:tasks.workunit.client.0.smithi040.stderr:++ awk '{print $1}' 2023-07-29T13:29:14.174 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91367/exe 2023-07-29T13:29:14.174 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:14.174 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:14.174 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:14.175 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:14.175 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:14.175 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:14.175 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:14.175 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:14.175 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:14.175 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:14.175 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:14.176 INFO:tasks.workunit.client.0.smithi040.stdout: test delete_pools_parallel on pid 91440 2023-07-29T13:29:14.177 INFO:tasks.workunit.client.0.smithi040.stderr:+ ff=delete_pools_parallel 2023-07-29T13:29:14.177 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91440 2023-07-29T13:29:14.177 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'test delete_pools_parallel on pid 91440' 2023-07-29T13:29:14.178 INFO:tasks.workunit.client.0.smithi040.stderr:+ pids[$f]=91440 2023-07-29T13:29:14.178 INFO:tasks.workunit.client.0.smithi040.stderr:+ ret=0 2023-07-29T13:29:14.178 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2023-07-29T13:29:14.178 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T13:29:14.178 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90784 2023-07-29T13:29:14.178 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90784 2023-07-29T13:29:14.178 INFO:tasks.workunit.client.0.smithi040.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2023-07-29T13:29:14.178 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_open_pools_parallel 2023-07-29T13:29:14.179 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -f /etc/bashrc ']' 2023-07-29T13:29:14.179 INFO:tasks.workunit.client.0.smithi040.stderr:+ . /etc/bashrc 2023-07-29T13:29:14.179 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -z '' ']' 2023-07-29T13:29:14.179 INFO:tasks.workunit.client.0.smithi040.stderr:++ BASHRCSOURCED=Y 2023-07-29T13:29:14.179 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.179 INFO:tasks.workunit.client.0.smithi040.stderr:++ shopt -q login_shell 2023-07-29T13:29:14.179 INFO:tasks.workunit.client.0.smithi040.stderr:+++ umask 2023-07-29T13:29:14.179 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2023-07-29T13:29:14.180 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' 0022 -eq 0 ']' 2023-07-29T13:29:14.180 INFO:tasks.workunit.client.0.smithi040.stderr:++ SHELL=/bin/bash 2023-07-29T13:29:14.180 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ open_pools_parallel: /' 2023-07-29T13:29:14.180 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.180 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-07-29T13:29:14.180 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.180 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/bash_completion.sh 2023-07-29T13:29:14.181 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-07-29T13:29:14.181 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.181 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-07-29T13:29:14.181 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.181 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/cobbler.sh 2023-07-29T13:29:14.181 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.181 INFO:tasks.workunit.client.0.smithi040.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-07-29T13:29:14.181 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.182 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-07-29T13:29:14.182 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.182 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorgrep.sh 2023-07-29T13:29:14.182 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.182 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: election_strategy: 1 2023-07-29T13:29:14.182 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:29:14.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:29:14.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:29:14.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:14.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 10 monclient: _finish_auth 0 2023-07-29T13:29:14.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 10 monclient: _check_auth_tickets 2023-07-29T13:29:14.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:14.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98015e40 con 0x5612b54ab210 2023-07-29T13:29:14.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:29:14.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98020640 con 0x5612b54ab210 2023-07-29T13:29:14.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:29:14.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:29:14.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba15f8640 10 monclient: dump: 2023-07-29T13:29:14.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: epoch 1 2023-07-29T13:29:14.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:29:14.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: min_mon_release 18 (reef) 2023-07-29T13:29:14.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: election_strategy: 1 2023-07-29T13:29:14.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:29:14.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:29:14.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:29:14.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:14.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba6a34780 5 monclient: authenticate success, global_id 4402 2023-07-29T13:29:14.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba6a34780 10 monclient: _renew_subs 2023-07-29T13:29:14.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba6a34780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:29:14.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5612b5333750 con 0x5612b54ab210 2023-07-29T13:29:14.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba6a34780 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:29:14.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba6a34780 10 monclient: _renew_subs 2023-07-29T13:29:14.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba6a34780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:29:14.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612b53331b0 con 0x5612b54ab210 2023-07-29T13:29:14.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba6a34780 1 librados: init done 2023-07-29T13:29:14.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba6a34780 10 librados: wait_for_osdmap waiting 2023-07-29T13:29:14.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.874+0000 7fdba25fa640 4 set_mon_vals no callback set 2023-07-29T13:29:14.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.926+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fdb980083c0 con 0x5612b54ab210 2023-07-29T13:29:14.189 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'grep=grep --color=auto' 2023-07-29T13:29:14.189 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'egrep=egrep --color=auto' 2023-07-29T13:29:14.189 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-07-29T13:29:14.189 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.189 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-07-29T13:29:14.189 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.190 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorls.sh 2023-07-29T13:29:14.190 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' '!' -t 0 ']' 2023-07-29T13:29:14.190 INFO:tasks.workunit.client.0.smithi040.stderr:+++ return 2023-07-29T13:29:14.190 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.190 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-07-29T13:29:14.190 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.190 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-07-29T13:29:14.190 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export S_COLORS=auto 2023-07-29T13:29:14.191 INFO:tasks.workunit.client.0.smithi040.stderr:+++ S_COLORS=auto 2023-07-29T13:29:14.191 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.191 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-07-29T13:29:14.191 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.191 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-07-29T13:29:14.191 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.191 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-07-29T13:29:14.191 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-07-29T13:29:14.192 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-07-29T13:29:14.192 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.192 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-07-29T13:29:14.192 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.192 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-07-29T13:29:14.192 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-07-29T13:29:14.192 INFO:tasks.workunit.client.0.smithi040.stderr:+++ /usr/libexec/grepconf.sh -c 2023-07-29T13:29:14.192 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-07-29T13:29:14.193 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-07-29T13:29:14.193 INFO:tasks.workunit.client.0.smithi040.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-07-29T13:29:14.193 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.193 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-07-29T13:29:14.193 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.193 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/debuginfod.sh 2023-07-29T13:29:14.193 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-07-29T13:29:14.193 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.194 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-07-29T13:29:14.194 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.194 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/gawk.sh 2023-07-29T13:29:14.194 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.194 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-07-29T13:29:14.194 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.194 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/lang.sh 2023-07-29T13:29:14.194 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup 2023-07-29T13:29:14.195 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/locale 2023-07-29T13:29:14.195 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.195 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG_backup=en_US.UTF-8 2023-07-29T13:29:14.195 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.195 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /etc/locale.conf ']' 2023-07-29T13:29:14.195 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -x /usr/bin/sed ']' 2023-07-29T13:29:14.195 INFO:tasks.workunit.client.0.smithi040.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-07-29T13:29:14.195 INFO:tasks.workunit.client.0.smithi040.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-07-29T13:29:14.196 INFO:tasks.workunit.client.0.smithi040.stderr:++++ export LANG=en_US.UTF-8 2023-07-29T13:29:14.196 INFO:tasks.workunit.client.0.smithi040.stderr:++++ LANG=en_US.UTF-8 2023-07-29T13:29:14.196 INFO:tasks.workunit.client.0.smithi040.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-07-29T13:29:14.196 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-07-29T13:29:14.196 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.196 INFO:tasks.workunit.client.0.smithi040.stderr:+++ LANG=en_US.UTF-8 2023-07-29T13:29:14.196 INFO:tasks.workunit.client.0.smithi040.stderr:+++ unset LANG_backup config 2023-07-29T13:29:14.197 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n '' ']' 2023-07-29T13:29:14.197 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-07-29T13:29:14.197 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' dumb = linux ']' 2023-07-29T13:29:14.197 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.197 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-07-29T13:29:14.197 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.197 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/less.sh 2023-07-29T13:29:14.197 INFO:tasks.workunit.client.0.smithi040.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-07-29T13:29:14.198 INFO:tasks.workunit.client.0.smithi040.stderr:++ for i in /etc/profile.d/*.sh 2023-07-29T13:29:14.198 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-07-29T13:29:14.198 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' ']' 2023-07-29T13:29:14.198 INFO:tasks.workunit.client.0.smithi040.stderr:++ . /etc/profile.d/which2.sh 2023-07-29T13:29:14.198 INFO:tasks.workunit.client.0.smithi040.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-07-29T13:29:14.198 INFO:tasks.workunit.client.0.smithi040.stderr:+++++ readlink /proc/91440/exe 2023-07-29T13:29:14.198 INFO:tasks.workunit.client.0.smithi040.stderr:++++ basename /usr/bin/bash 2023-07-29T13:29:14.198 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_declare='declare -f' 2023-07-29T13:29:14.199 INFO:tasks.workunit.client.0.smithi040.stderr:+++ which_opt=-f 2023-07-29T13:29:14.199 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export which_declare 2023-07-29T13:29:14.199 INFO:tasks.workunit.client.0.smithi040.stderr:+++ export -f which 2023-07-29T13:29:14.199 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset i 2023-07-29T13:29:14.199 INFO:tasks.workunit.client.0.smithi040.stderr:++ unset -f pathmunge 2023-07-29T13:29:14.199 INFO:tasks.workunit.client.0.smithi040.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-07-29T13:29:14.199 INFO:tasks.workunit.client.0.smithi040.stderr:+ export PATH 2023-07-29T13:29:14.199 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-07-29T13:29:14.200 INFO:tasks.workunit.client.0.smithi040.stderr:+ unset rc 2023-07-29T13:29:14.200 INFO:tasks.workunit.client.0.smithi040.stderr:+ ceph_test_rados_delete_pools_parallel 2023-07-29T13:29:14.200 INFO:tasks.workunit.client.0.smithi040.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2023-07-29T13:29:14.200 INFO:tasks.workunit.client.0.smithi040.stderr:+ sed 's/^/ delete_pools_parallel: /' 2023-07-29T13:29:14.201 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.927+0000 7fdba15f8640 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fdb9006a1d0 0x7fdb9006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=:13.747+0000 7f9a0b838780 10 monclient(hunting): authenticate will time out at 1176.880666s 2023-07-29T13:29:14.201 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.761+0000 7f9a03fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x557ec1a05a00 0x557ec1a985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:29:14.201 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.761+0000 7f9a03fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x557ec1a05a00 0x557ec1a985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54940/0 (socket says 172.21.15.40:54940) 2023-07-29T13:29:14.201 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.761+0000 7f9a03fff640 1 -- 172.21.15.40:0/3674074507 learned_addr learned my addr 172.21.15.40:0/3674074507 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:29:14.201 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.761+0000 7f9a03fff640 10 monclient(hunting): get_auth_request con 0x557ec1a05a00 auth_method 0 2023-07-29T13:29:14.202 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.761+0000 7f9a03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:29:14.202 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.761+0000 7f9a03fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:29:14.202 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.761+0000 7f9a03fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:29:14.202 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.763+0000 7f9a03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:29:14.202 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.763+0000 7f9a03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:29:14.202 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.763+0000 7f9a03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:29:14.202 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.826+0000 7f9a03fff640 10 monclient(hunting): handle_auth_done global_id 4404 payload 274 2023-07-29T13:29:14.202 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.826+0000 7f9a03fff640 1 -- 172.21.15.40:0/3674074507 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x557ec1a98ae0 msgr2=0x557ec1a03760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T13:29:14.203 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.826+0000 7f9a03fff640 1 --2- 172.21.15.40:0/3674074507 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x557ec1a98ae0 0x557ec1a03760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.203 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.826+0000 7f9a03fff640 1 -- 172.21.15.40:0/3674074507 >> v1:172.21.15.40:6789/0 conn(0x557ec1a03ca0 legacy=0x557ec19fea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:29:14.203 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.826+0000 7f9a03fff640 10 monclient: _finish_hunting 0 2023-07-29T13:29:14.203 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.826+0000 7f9a03fff640 1 monclient: found mon.noname-a 2023-07-29T13:29:14.203 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.826+0000 7f9a03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:29:14.203 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.826+0000 7f9a03fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T13:29:14.203 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.826+0000 7f9a03fff640 1 -- 172.21.15.40:0/3674074507 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ec0ddd4b0 con 0x557ec1a05a00 2023-07-29T13:29:14.203 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.924+0000 7f9a03fff640 1 --2- 172.21.15.40:0/3674074507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ec1a05a00 0x557ec1a985a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f99f400a5a0 tx=0x7f99f401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc677879e8920fb6 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:29:14.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 1 -- 172.21.15.40:0/3674074507 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f404a8f0 con 0x557ec1a05a00 2023-07-29T13:29:14.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:29:14.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T13:29:14.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 10 monclient: dump: 2023-07-29T13:29:14.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: epoch 1 2023-07-29T13:29:14.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:29:14.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: min_mon_release 18 (reef) 2023-07-29T13:29:14.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: election_strategy: 1 2023-07-29T13:29:14.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:29:14.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:29:14.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:29:14.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 10 monclient: _finish_auth 0 2023-07-29T13:29:14.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 10 monclient: _check_auth_tickets 2023-07-29T13:29:14.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:14.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 1 -- 172.21.15.40:0/3674074507 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f404aa50 con 0x557ec1a05a00 2023-07-29T13:29:14.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:29:14.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 1 -- 172.21.15.40:0/3674074507 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f404ade0 con 0x557ec1a05a00 2023-07-29T13:29:14.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:29:14.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T13:29:14.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a02ffd640 10 monclient: dump: 2023-07-29T13:29:14.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: epoch 1 2023-07-29T13:29:14.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:29:14.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: min_mon_release 18 (reef) 2023-07-29T13:29:14.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: election_strategy: 1 2023-07-29T13:29:14.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:29:14.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:29:14.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:29:14.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a0b838780 5 monclient: authenticate success, global_id 4404 2023-07-29T13:29:14.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a0b838780 10 monclient: get_monmap_and_config success 2023-07-29T13:29:14.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.958+0000 7f9a0b838780 4 set_mon_vals no callback set 2023-07-29T13:29:14.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.959+0000 7f9a0b838780 10 monclient: shutdown 2023-07-29T13:29:14.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.959+0000 7f9a0b838780 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:29:14.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.959+0000 7f9a0b838780 1 -- 172.21.15.40:0/3674074507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ec1a05a00 msgr2=0x557ec1a985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:29:14.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.959+0000 7f9a0b838780 1 --2- 172.21.15.40:0/3674074507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ec1a05a00 0x557ec1a985a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f99f400a5a0 tx=0x7f99f401cc60 comp rx=0 tx=0).stop 2023-07-29T13:29:14.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.959+0000 7f9a0b838780 1 -- 172.21.15.40:0/3674074507 shutdown_connections 2023-07-29T13:29:14.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.959+0000 7f9a0b838780 1 --2- 172.21.15.40:0/3674074507 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x557ec1a98ae0 0x557ec1a03760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.959+0000 7f9a0b838780 1 --2- 172.21.15.40:0/3674074507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ec1a05a00 0x557ec1a985a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.959+0000 7f9a0b838780 1 -- 172.21.15.40:0/3674074507 >> 172.21.15.40:0/3674074507 conn(0x557ec19be570 msgr2=0x557ec19c7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:29:14.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.959+0000 7f9a0b838780 1 -- 172.21.15.40:0/3674074507 shutdown_connections 2023-07-29T13:29:14.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.959+0000 7f9a0b838780 1 -- 172.21.15.40:0/3674074507 wait complete. 2023-07-29T13:29:14.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.960+0000 7f9a0b838780 -1 asok(0x557ec0e5f370) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.admin.90894.asok': (17) File exists 2023-07-29T13:29:14.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.960+0000 7f9a0b838780 10 monclient: build_initial_monmap 2023-07-29T13:29:14.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.960+0000 7f9a0b838780 10 monclient: monmap: 2023-07-29T13:29:14.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: epoch 0 2023-07-29T13:29:14.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: fsid 00000000-0000-0000-0000-000000000000 2023-07-29T13:29:14.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: last_changed 0.000000 2023-07-29T13:29:14.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: created 0.000000 2023-07-29T13:29:14.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: min_mon_release 0 (unknown) 2023-07-29T13:29:14.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: election_strategy: 1 2023-07-29T13:29:14.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T13:29:14.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T13:29:14.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T13:29:14.212 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.212 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.961+0000 7f9a0b838780 1 librados: starting msgr at 2023-07-29T13:29:14.212 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.961+0000 7f9a0b838780 1 librados: starting objecter 2023-07-29T13:29:14.212 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.961+0000 7f9a0b838780 1 Processor -- start 2023-07-29T13:29:14.212 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.961+0000 7f9a0b838780 1 -- start start 2023-07-29T13:29:14.212 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.961+0000 7f9a0b838780 1 librados: setting wanted keys 2023-07-29T13:29:14.212 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.961+0000 7f9a0b838780 1 librados: calling monclient init 2023-07-29T13:29:14.213 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.961+0000 7f9a0b838780 10 monclient: init 2023-07-29T13:29:14.213 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 10 monclient: _reopen_session rank -1 2023-07-29T13:29:14.213 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T13:29:14.213 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ec1a03ca0 0x557ec1a99190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:14.213 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 10 monclient(hunting): picked mon.noname-a con 0x557ec1a03ca0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T13:29:14.213 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x557ec1a05a00 0x557ec1a07270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:14.213 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 10 monclient(hunting): picked mon.noname-c con 0x557ec1a05a00 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T13:29:14.213 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557ec1a98ae0 0x557ec1a9ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:14.214 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 10 monclient(hunting): picked mon.noname-b con 0x557ec1a98ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T13:29:14.214 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 10 monclient(hunting): start opening mon connection 2023-07-29T13:29:14.214 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ec0db3fa0 con 0x557ec1a03ca0 2023-07-29T13:29:14.214 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 10 monclient(hunting): start opening mon connection 2023-07-29T13:29:14.214 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ec0d973f0 con 0x557ec1a98ae0 2023-07-29T13:29:14.214 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 10 monclient(hunting): start opening mon connection 2023-07-29T13:29:14.214 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ec1a92750 con 0x557ec1a05a00 2023-07-29T13:29:14.215 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 10 monclient(hunting): _renew_subs 2023-07-29T13:29:14.215 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.962+0000 7f9a0b838780 10 monclient(hunting): authenticate will time out at 1177.094727s 2023-07-29T13:29:14.215 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.982+0000 7f9a08c36640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557ec1a98ae0 0x557ec1a9ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:29:14.215 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.992+0000 7f9a08c36640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557ec1a98ae0 0x557ec1a9ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:48960/0 (socket says 172.21.15.40:48960) 2023-07-29T13:29:14.215 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.992+0000 7f9a08c36640 1 -- 172.21.15.40:0/207577766 learned_addr learned my addr 172.21.15.40:0/207577766 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T13:29:14.215 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.992+0000 7f9a08c36640 10 monclient(hunting): get_auth_request con 0x557ec1a98ae0 auth_method 0 2023-07-29T13:29:14.215 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.992+0000 7f9a08c36640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:29:14.215 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.992+0000 7f9a08c36640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:29:14.216 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.992+0000 7f9a08c36640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:29:14.216 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.992+0000 7f9a03fff640 1 --2- 172.21.15.40:0/207577766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ec1a03ca0 0x557ec1a99190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:29:14.216 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.992+0000 7f9a03fff640 10 monclient(hunting): get_auth_request con 0x557ec1a03ca0 auth_method 0 2023-07-29T13:29:14.216 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.992+0000 7f9a03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T13:29:14.216 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.992+0000 7f9a03fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T13:29:14.216 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.992+0000 7f9a03fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T13:29:14.216 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.993+0000 7f9a03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:29:14.217 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.993+0000 7f9a03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:29:14.217 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.993+0000 7f9a03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:29:14.217 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.993+0000 7f9a08c36640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T13:29:14.217 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.993+0000 7f9a08c36640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T13:29:14.217 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:13.993+0000 7f9a08c36640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T13:29:14.217 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.011+0000 7f9a08c36640 10 monclient(hunting): handle_auth_done global_id 4438 payload 824 2023-07-29T13:29:14.217 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.011+0000 7f9a08c36640 1 -- 172.21.15.40:0/207577766 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x557ec1a05a00 msgr2=0x557ec1a07270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:29:14.217 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.011+0000 7f9a08c36640 1 --2- 172.21.15.40:0/207577766 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x557ec1a05a00 0x557ec1a07270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.218 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.011+0000 7f9a08c36640 1 -- 172.21.15.40:0/207577766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ec1a03ca0 msgr2=0x557ec1a99190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:29:14.218 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.011+0000 7f9a08c36640 1 --2- 172.21.15.40:0/207577766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ec1a03ca0 0x557ec1a99190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:14.218 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.011+0000 7f9a08c36640 10 monclient: _finish_hunting 0 2023-07-29T13:29:14.218 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.011+0000 7f9a08c36640 1 monclient: found mon.noname-b 2023-07-29T13:29:14.218 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.011+0000 7f9a08c36640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:29:14.218 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.011+0000 7f9a08c36640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T13:29:14.218 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.011+0000 7f9a08c36640 1 -- 172.21.15.40:0/207577766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f4005de0 con 0x557ec1a98ae0 2023-07-29T13:29:14.218 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.034+0000 7f9a08c36640 1 --2- 172.21.15.40:0/207577766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557ec1a98ae0 0x557ec1a9ba30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f99fc002190 tx=0x7f99fc00b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:29:14.219 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.034+0000 7f9a00ff9640 10 client.?.objecter ms_handle_connect 0x557ec1a98ae0 2023-07-29T13:29:14.219 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.034+0000 7f9a00ff9640 10 client.?.objecter resend_mon_ops 2023-07-29T13:29:14.219 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 1 -- 172.21.15.40:0/207577766 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc006220 con 0x557ec1a98ae0 2023-07-29T13:29:14.219 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:29:14.219 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T13:29:14.219 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 10 monclient: dump: 2023-07-29T13:29:14.219 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: epoch 1 2023-07-29T13:29:14.220 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:29:14.220 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.220 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.220 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: min_mon_release 18 (reef) 2023-07-29T13:29:14.220 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: election_strategy: 1 2023-07-29T13:29:14.220 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:29:14.220 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:29:14.220 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:29:14.221 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.221 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 10 monclient: _finish_auth 0 2023-07-29T13:29:14.221 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 10 monclient: _check_auth_tickets 2023-07-29T13:29:14.221 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:14.221 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 1 -- 172.21.15.40:0/207577766 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc005440 con 0x557ec1a98ae0 2023-07-29T13:29:14.221 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a0b838780 5 monclient: authenticate success, global_id 4438 2023-07-29T13:29:14.221 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a0b838780 10 monclient: _renew_subs 2023-07-29T13:29:14.222 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a0b838780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:29:14.222 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a0b838780 1 -- 172.21.15.40:0/207577766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557ec0dc3310 con 0x557ec1a98ae0 2023-07-29T13:29:14.222 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T13:29:14.222 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 1 -- 172.21.15.40:0/207577766 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0057f0 con 0x557ec1a98ae0 2023-07-29T13:29:14.222 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T13:29:14.222 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T13:29:14.222 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a00ff9640 10 monclient: dump: 2023-07-29T13:29:14.222 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: epoch 1 2023-07-29T13:29:14.223 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:29:14.223 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.223 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: created 2023-07-29T13:26:33.423641+0000 2023-07-29T13:29:14.223 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: min_mon_release 18 (reef) 2023-07-29T13:29:14.223 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: election_strategy: 1 2023-07-29T13:29:14.223 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T13:29:14.223 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T13:29:14.223 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T13:29:14.224 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.224 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a0b838780 10 client.4438.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:29:14.224 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a0b838780 10 monclient: _renew_subs 2023-07-29T13:29:14.224 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a0b838780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:29:14.224 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a0b838780 1 -- 172.21.15.40:0/207577766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ec0db6750 con 0x557ec1a98ae0 2023-07-29T13:29:14.224 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a0b838780 1 librados: init done 2023-07-29T13:29:14.224 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a0b838780 10 librados: watch_flush enter 2023-07-29T13:29:14.224 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a027fc640 4 set_mon_vals no callback set 2023-07-29T13:29:14.225 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.087+0000 7f9a0b838780 10 librados: watch_flush exit 2023-07-29T13:29:14.225 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 20 client.4438.objecter shutdown clearing up homeless session... 2023-07-29T13:29:14.225 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 10 client.4438.objecter successfully canceled tick 2023-07-29T13:29:14.225 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 10 monclient: shutdown 2023-07-29T13:29:14.225 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:29:14.225 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 1 -- 172.21.15.40:0/207577766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557ec1a98ae0 msgr2=0x557ec1a9ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:29:14.225 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 1 --2- 172.21.15.40:0/207577766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557ec1a98ae0 0x557ec1a9ba30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f99fc002190 tx=0x7f99fc00b570 comp rx=0 tx=0).stop 2023-07-29T13:29:16.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+00000 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:16.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.932+0000 7fdba3dfd640 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fdb9006a1d0 0x7fdb9006c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:29:16.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.932+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdb9805c400 con 0x5612b54ab210 2023-07-29T13:29:16.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.932+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(24..24 src has 1..24) v4 2023-07-29T13:29:16.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.932+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [24,24] > 0 2023-07-29T13:29:16.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.932+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding full epoch 24 2023-07-29T13:29:16.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.932+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:16.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.932+0000 7fdba6a34780 10 librados: wait_for_osdmap done waiting 2023-07-29T13:29:16.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.932+0000 7fdba6a34780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:29:16.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.932+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb9404cd00 con 0x5612b54ab210 2023-07-29T13:29:16.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.936+0000 7fdba3dfd640 10 monclient: get_auth_request con 0x7fdb9006a1d0 auth_method 0 2023-07-29T13:29:16.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.939+0000 7fdba3dfd640 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fdb9006a1d0 0x7fdb9006c680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5612b5421bb0 tx=0x7fdb9404e070 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:29:16.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.939+0000 7fdba15f8640 10 client.4402.objecter ms_handle_connect 0x7fdb9006a1d0 2023-07-29T13:29:16.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.979+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb98058210 con 0x5612b54ab210 2023-07-29T13:29:16.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.979+0000 7fdba15f8640 10 monclient: handle_get_version_reply finishing 1 version 24 2023-07-29T13:29:16.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.979+0000 7fdba1df9640 10 client.4402.objecter _get_latest_version latest 24, have it 2023-07-29T13:29:16.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.979+0000 7fdba6a34780 10 client.4402.objecter create_pool name=ceph_test_rados_api_asio 2023-07-29T13:29:16.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.979+0000 7fdba6a34780 10 client.4402.objecter pool_op_submit 1 2023-07-29T13:29:16.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.979+0000 7fdba6a34780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:29:16.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:13.979+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) v4 -- 0x5612b539f7b0 con 0x5612b54ab210 2023-07-29T13:29:16.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.689+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v26) v1 ==== 43+0+0 (secure 0 0 0) 0x7fdb98025020 con 0x5612b54ab210 2023-07-29T13:29:16.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.689+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 pool_op_reply(tid 1 (0) Success v26) v1 2023-07-29T13:29:16.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.689+0000 7fdba15f8640 10 client.4402.objecter handle_pool_op_reply pool_op_reply(tid 1 (0) Success v26) v1 2023-07-29T13:29:16.824 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.689+0000 7fdba15f8640 10 client.4402.objecter have request 1 at 0x5612b54a6250 Op: create 2023-07-29T13:29:16.824 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.689+0000 7fdba15f8640 20 client.4402.objecter waiting for client to reach epoch 26 before calling back 2023-07-29T13:29:16.824 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.689+0000 7fdba15f8640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:29:16.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.689+0000 7fdba15f8640 10 monclient: _renew_subs 2023-07-29T13:29:16.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.689+0000 7fdba15f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:29:16.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.689+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=25}) v3 -- 0x7fdb9006ff00 con 0x5612b54ab210 2023-07-29T13:29:16.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.689+0000 7fdba15f8640 10 client.4402.objecter done 2023-07-29T13:29:16.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 8 ==== osd_map(25..26 src has 1..26) v4 ==== 11180+0+0 (secure 0 0 0) 0x7fdb98060790 con 0x5612b54ab210 2023-07-29T13:29:16.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(25..26 src has 1..26) v4 2023-07-29T13:29:16.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [25,26] > 24 2023-07-29T13:29:16.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 25 2023-07-29T13:29:16.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 26 2023-07-29T13:29:16.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:16.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba6a34780 10 librados: set snap read head -> 1 2023-07-29T13:29:16.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba6a34780 10 librados: writefull oid=exist nspace= 2023-07-29T13:29:16.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b54a5c10 2023-07-29T13:29:16.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 26 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:16.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:16.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:16.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.742+0000 7fdba6a34780 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x5612b54a4ec0 0x5612b5499320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T13:29:16.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.743+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:16.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.743+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [writefull 0~5 in=5b] tid 2 osd.2 2023-07-29T13:29:16.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.743+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:16.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.743+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 2 2023-07-29T13:29:16.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.743+0000 7fdba6a34780 15 client.4402.objecter _send_op 2 to 7.1 on osd.2 2023-07-29T13:29:16.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.743+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:2 7.1 7:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26) v8 -- 0x5612b5499860 con 0x5612b54a4ec0 2023-07-29T13:29:16.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.743+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:16.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.743+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:29:24.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.823+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:24.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.823+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:24.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.823+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:24.825 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:16.823+0000 7fdb8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:29:24.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:17.823+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:24.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:17.823+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:24.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:17.823+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:24.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:18.823+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:24.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:18.823+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:24.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:18.823+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:24.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:18.875+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:29:24.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:18.997+0000 7fdba45fe640 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x5612b54a4ec0 0x5612b5499320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T13:29:24.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:18.997+0000 7fdba45fe640 10 monclient: get_auth_request con 0x5612b54a4ec0 auth_method 0 2023-07-29T13:29:24.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:19.002+0000 7fdba15f8640 10 client.4402.objecter ms_handle_connect 0x5612b54a4ec0 2023-07-29T13:29:24.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:19.002+0000 7fdba45fe640 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x5612b54a4ec0 0x5612b5499320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T13:29:24.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:19.823+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:24.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:19.823+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:24.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:19.823+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:24.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:20.823+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:24.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:20.823+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:24.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:20.823+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:24.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:21.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:24.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:21.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:24.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:21.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:24.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:22.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:24.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:22.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:24.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:22.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:24.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:23.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:24.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:23.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:24.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:23.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:24.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:23.876+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:29:24.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:24.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:24.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:24.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:26.037 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:02:21.371 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:26.041 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:02:21.375 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:26.106 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:02:21.390 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:26.117 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:02:21.401 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:26.123 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:02:21.457 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:26.129 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:02:21.463 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:26.183 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:02:21.517 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:26.230 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:02:21.514 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:26.259 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:02:21.594 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:28.976 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:24.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:28.977 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:25.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:28.977 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:25.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:28.977 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:25.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:28.977 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:26.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:28.977 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:26.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:28.977 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:26.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:28.978 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:26.824+0000 7fdb8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:29:28.978 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:27.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:28.978 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:27.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:28.978 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:27.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:28.978 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.823+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:28.978 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.823+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:28.978 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.823+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:28.979 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.876+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:29:28.979 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.876+0000 7fdba6061640 2 client.4402.objecter tid 2 on osd.2 is laggy 2023-07-29T13:29:28.979 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.876+0000 7fdba6061640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:29:28.979 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.876+0000 7fdba6061640 10 monclient: _renew_subs 2023-07-29T13:29:28.979 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.876+0000 7fdba6061640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:29:28.979 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.876+0000 7fdba6061640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=27}) v3 -- 0x7fdb74002f90 con 0x5612b54ab210 2023-07-29T13:29:28.979 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.876+0000 7fdba6061640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- ping magic: 0 v1 -- 0x7fdb74003320 con 0x5612b54a4ec0 2023-07-29T13:29:28.980 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 9 ==== osd_map(27..35 src has 1..35) v4 ==== 14020+0+0 (secure 0 0 0) 0x7fdb98005430 con 0x5612b54ab210 2023-07-29T13:29:28.980 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(27..35 src has 1..35) v4 2023-07-29T13:29:28.980 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [27,35] > 26 2023-07-29T13:29:28.980 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 27 2023-07-29T13:29:28.980 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:28.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 27 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:28.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:28.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 28 2023-07-29T13:29:28.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:28.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 28 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:28.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:28.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 29 2023-07-29T13:29:28.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:28.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 29 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:28.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:28.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.974+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 30 2023-07-29T13:29:28.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:28.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 30 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:28.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:28.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 31 2023-07-29T13:29:28.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:28.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 31 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:28.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:28.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 32 2023-07-29T13:29:28.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:28.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 32 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:28.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:28.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 33 2023-07-29T13:29:28.984 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:28.984 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 33 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:28.984 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:28.984 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 34 2023-07-29T13:29:28.984 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:28.984 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 34 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:28.984 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:28.985 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 35 2023-07-29T13:29:28.985 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:28.985 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 35 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:28.985 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:28.985 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:34.007 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:28.975+0000 7fdba15f8640 20 client.4402.objecter 2 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:29:34.008 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:29.823+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:34.008 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:29.823+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:34.008 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:29.823+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:34.008 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:30.823+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:34.008 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:30.823+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:34.008 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:30.823+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:34.009 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:31.823+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:34.009 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:31.823+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:34.009 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:31.823+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:34.009 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:32.823+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:34.009 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:32.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:34.009 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:32.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:34.009 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:33.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:34.009 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:33.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:34.010 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:33.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:34.010 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:33.877+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:29:34.010 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:33.877+0000 7fdba6061640 2 client.4402.objecter tid 2 on osd.2 is laggy 2023-07-29T13:29:34.010 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:33.877+0000 7fdba6061640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:29:34.010 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:33.877+0000 7fdba6061640 10 monclient: _renew_subs 2023-07-29T13:29:34.010 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:33.877+0000 7fdba6061640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:29:34.010 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:33.877+0000 7fdba6061640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=36}) v3 -- 0x7fdb740034e0 con 0x5612b54ab210 2023-07-29T13:29:34.010 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:33.877+0000 7fdba6061640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- ping magic: 0 v1 -- 0x7fdb740036d0 con 0x5612b54a4ec0 2023-07-29T13:29:34.011 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.005+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 10 ==== osd_map(36..39 src has 1..39) v4 ==== 2893+0+0 (secure 0 0 0) 0x7fdb9805c130 con 0x5612b54ab210 2023-07-29T13:29:34.011 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.005+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(36..39 src has 1..39) v4 2023-07-29T13:29:34.011 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.005+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [36,39] > 35 2023-07-29T13:29:34.011 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.005+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 36 2023-07-29T13:29:34.011 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:34.011 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 36 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:34.011 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:34.012 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 37 2023-07-29T13:29:34.012 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:38.879 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 37 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:38.880 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:38.880 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 38 2023-07-29T13:29:38.880 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:38.880 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 38 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:38.880 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:38.881 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 39 2023-07-29T13:29:38.881 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:38.881 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 39 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:38.881 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:38.881 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:38.881 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.006+0000 7fdba15f8640 20 client.4402.objecter 2 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:29:38.881 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:38.881 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:38.882 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:34.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:38.882 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:35.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:38.882 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:35.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:38.882 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:35.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:38.882 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:36.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:38.882 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:36.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:38.882 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:36.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:38.883 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:36.824+0000 7fdb8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:29:38.883 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:37.345+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 1 ==== osd_map(27..37 src has 1..37) v4 ==== 15638+0+0 (crc 0 0 0) 0x7fdb9c006be0 con 0x5612b54a4ec0 2023-07-29T13:29:38.883 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:37.345+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(27..37 src has 1..37) v4 2023-07-29T13:29:38.883 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:37.345+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map ignoring epochs [27,37] <= 39 2023-07-29T13:29:38.883 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:37.345+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:38.883 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:37.345+0000 7fdba15f8640 20 client.4402.objecter 2 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:29:38.883 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:37.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:38.883 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:37.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:38.884 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:37.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:38.884 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:38.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:38.884 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:38.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:38.884 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:38.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:40.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:38.878+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:29:40.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:38.878+0000 7fdba6061640 2 client.4402.objecter tid 2 on osd.2 is laggy 2023-07-29T13:29:40.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:38.878+0000 7fdba6061640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:29:40.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:38.878+0000 7fdba6061640 10 monclient: _renew_subs 2023-07-29T13:29:40.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:38.878+0000 7fdba6061640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:29:40.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:38.878+0000 7fdba6061640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=40}) v3 -- 0x7fdb74003890 con 0x5612b54ab210 2023-07-29T13:29:40.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:38.878+0000 7fdba6061640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- ping magic: 0 v1 -- 0x7fdb74003ae0 con 0x5612b54a4ec0 2023-07-29T13:29:40.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 11 ==== osd_map(40..42 src has 1..42) v4 ==== 1204+0+0 (secure 0 0 0) 0x7fdb98054b50 con 0x5612b54ab210 2023-07-29T13:29:40.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(40..42 src has 1..42) v4 2023-07-29T13:29:40.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [40,42] > 39 2023-07-29T13:29:40.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 40 2023-07-29T13:29:40.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:40.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 40 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:40.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:40.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 41 2023-07-29T13:29:40.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:40.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 41 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:40.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:40.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 42 2023-07-29T13:29:40.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 10 client.4402.objecter checking op 2 2023-07-29T13:29:40.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 42 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:40.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:40.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:40.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.098+0000 7fdba15f8640 20 client.4402.objecter 2 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:29:40.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:40.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:40.833 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:39.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:40.833 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:40.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:40.833 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:40.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:42.733 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:40.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by api_service: [==========] Running 4 tests from 1 test suite. 2023-07-29T13:29:42.733 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [----------] Global test environment set-up. 2023-07-29T13:29:42.734 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [----------] 4 tests from LibRadosService 2023-07-29T13:29:42.734 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2023-07-29T13:29:42.734 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5554 ms) 2023-07-29T13:29:42.734 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2023-07-29T13:29:42.734 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [ OK ] LibRadosService.RegisterLate (127 ms) 2023-07-29T13:29:42.734 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2023-07-29T13:29:42.735 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: cluster: 2023-07-29T13:29:42.735 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: id: d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:29:42.735 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: health: HEALTH_OK 2023-07-29T13:29:42.735 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 2023-07-29T13:29:42.735 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: services: 2023-07-29T13:29:42.735 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 2m) 2023-07-29T13:29:42.735 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: mgr: x(active, since 111s), standbys: y 2023-07-29T13:29:42.736 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: osd: 8 osds: 8 up (since 73s), 8 in (since 117s) 2023-07-29T13:29:42.736 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: laundry: 2 daemons active (1 hosts) 2023-07-29T13:29:42.736 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 2023-07-29T13:29:42.736 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: data: 2023-07-29T13:29:42.736 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: pools: 27 pools, 809 pgs 2023-07-29T13:29:42.736 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: objects: 2 objects, 449 KiB 2023-07-29T13:29:42.736 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: usage: 214 MiB used, 720 GiB / 720 GiB avail 2023-07-29T13:29:42.736 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: pgs: 95.303% pgs unknown 2023-07-29T13:29:42.737 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 3.585% pgs not active 2023-07-29T13:29:42.737 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 771 unknown 2023-07-29T13:29:42.737 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 29 creating+peering 2023-07-29T13:29:42.737 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 9 active+clean 2023-07-29T13:29:42.737 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 2023-07-29T13:29:42.737 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 2023-07-29T13:29:42.738 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: cluster: 2023-07-29T13:29:42.738 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: id: d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T13:29:42.738 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: health: HEALTH_OK 2023-07-29T13:29:42.738 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 2023-07-29T13:29:42.738 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: services: 2023-07-29T13:29:42.738 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 2m) 2023-07-29T13:29:42.739 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: mgr: x(active, since 113s), standbys: y 2023-07-29T13:29:42.739 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: osd: 8 osds: 8 up (since 76s), 8 in (since 119s) 2023-07-29T13:29:42.739 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2023-07-29T13:29:42.739 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: laundry: 1 daemon active (1 hosts) 2023-07-29T13:29:42.739 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 2023-07-29T13:29:42.739 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: data: 2023-07-29T13:29:42.739 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: pools: 27 pools, 785 pgs 2023-07-29T13:29:42.739 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: objects: 2 objects, 449 KiB 2023-07-29T13:29:42.740 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: usage: 213 MiB used, 720 GiB / 720 GiB avail 2023-07-29T13:29:42.740 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: pgs: 26.879% pgs unknown 2023-07-29T13:29:42.740 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 70.955% pgs not active 2023-07-29T13:29:42.740 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 512 creating+peering 2023-07-29T13:29:42.740 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 211 unknown 2023-07-29T13:29:42.740 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 45 creating+activating 2023-07-29T13:29:42.740 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 17 active+clean 2023-07-29T13:29:42.741 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 2023-07-29T13:29:42.741 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 2023-07-29T13:29:42.741 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [ OK ] LibRadosService.StatusFormat (3111 ms) 2023-07-29T13:29:42.741 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [ RUN ] LibRadosService.Status 2023-07-29T13:29:42.741 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [ OK ] LibRadosService.Status (20068 ms) 2023-07-29T13:29:42.741 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [----------] 4 tests from LibRadosService (28860 ms total) 2023-07-29T13:29:42.741 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: 2023-07-29T13:29:42.741 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [----------] Global test environment tear-down 2023-07-29T13:29:42.742 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [==========] 4 tests from 1 test suite ran. (28860 ms total) 2023-07-29T13:29:42.742 INFO:tasks.workunit.client.0.smithi040.stdout: api_service: [ PASSED ] 4 tests. 2023-07-29T13:29:43.292 INFO:tasks.workunit.client.0.smithi040.stdout: client.admin 2023-07-29T13:29:43.293 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:41.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:43.293 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:41.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:43.293 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:41.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:43.293 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:42.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:43.293 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:42.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:43.293 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:42.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:43.293 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 2 ==== osd_op_reply(2 exist [writefull 0~5] v37'1 uv1 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:43.294 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(2 exist [writefull 0~5] v37'1 uv1 ondisk = 0) v8 2023-07-29T13:29:43.294 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:43.294 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 2 ondisk uv 1 in 7.1 attempt 0 2023-07-29T13:29:43.294 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:29:43.294 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 2 2023-07-29T13:29:43.294 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba45fe640 15 client.4402.objecter _finish_op 2 2023-07-29T13:29:43.294 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:43.294 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 2 2023-07-29T13:29:43.295 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:29:43.295 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 10 librados: Objecter returned from writefull r=0 2023-07-29T13:29:43.295 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2023-07-29T13:29:43.295 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b54a5c10 2023-07-29T13:29:43.295 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 42 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:43.295 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:43.295 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:43.296 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:43.296 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [read 0~256] tid 3 osd.2 2023-07-29T13:29:43.296 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:43.296 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 3 2023-07-29T13:29:43.296 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 15 client.4402.objecter _send_op 3 to 7.1 on osd.2 2023-07-29T13:29:43.296 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:3 7.1 7:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x5612b54a6380 con 0x5612b54a4ec0 2023-07-29T13:29:43.296 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:43.297 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:29:44.622 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29: api_service_pp: [==========] Running 4 tests from 1 test suite. 2023-07-29T13:29:44.622 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [----------] Global test environment set-up. 2023-07-29T13:29:44.623 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2023-07-29T13:29:44.623 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2023-07-29T13:29:44.623 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5381 ms) 2023-07-29T13:29:44.623 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2023-07-29T13:29:44.623 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (96 ms) 2023-07-29T13:29:44.623 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2023-07-29T13:29:44.623 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20057 ms) 2023-07-29T13:29:44.624 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2023-07-29T13:29:44.624 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: attempt 0 of 20 2023-07-29T13:29:44.624 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: failed to deregister: 2023-07-29T13:29:44.624 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: { 2023-07-29T13:29:44.624 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "epoch": 10, 2023-07-29T13:29:44.624 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "modified": "2023-07-29T13:29:41.038693+0000", 2023-07-29T13:29:44.624 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "services": { 2023-07-29T13:29:44.625 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "laundry": { 2023-07-29T13:29:44.625 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "daemons": { 2023-07-29T13:29:44.625 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "summary": "", 2023-07-29T13:29:44.625 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "close-test-pid91371": { 2023-07-29T13:29:44.625 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "start_epoch": 10, 2023-07-29T13:29:44.625 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "start_stamp": "2023-07-29T13:29:39.523138+0000", 2023-07-29T13:29:44.625 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "gid": 4502, 2023-07-29T13:29:44.625 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "addr": "172.21.15.40:0/1965872371", 2023-07-29T13:29:44.626 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "metadata": { 2023-07-29T13:29:44.626 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "arch": "x86_64", 2023-07-29T13:29:44.626 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "ceph_release": "reef", 2023-07-29T13:29:44.626 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "ceph_version": "ceph version 18.1.3-11-g0484097b (0484097b0572e96f42cf6402cbe0ac8dcb046577) reef (rc)", 2023-07-29T13:29:44.626 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "ceph_version_short": "18.1.3-11-g0484097b", 2023-07-29T13:29:44.626 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2023-07-29T13:29:44.626 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "distro": "centos", 2023-07-29T13:29:44.627 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2023-07-29T13:29:44.627 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "distro_version": "9", 2023-07-29T13:29:44.627 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "foo": "bar", 2023-07-29T13:29:44.627 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "hostname": "smithi040", 2023-07-29T13:29:44.627 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Mon Jul 24 09:26:29 UTC 2023", 2023-07-29T13:29:44.627 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "kernel_version": "5.14.0-344.el9.x86_64", 2023-07-29T13:29:44.627 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "mem_swap_kb": "0", 2023-07-29T13:29:44.627 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "mem_total_kb": "32495004", 2023-07-29T13:29:44.628 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "os": "Linux", 2023-07-29T13:29:44.628 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "this": "that" 2023-07-29T13:29:44.628 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: }, 2023-07-29T13:29:44.628 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "task_status": {} 2023-07-29T13:29:44.628 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: }, 2023-07-29T13:29:44.628 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "pid91234": { 2023-07-29T13:29:44.629 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "start_epoch": 9, 2023-07-29T13:29:44.629 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "start_stamp": "2023-07-29T13:29:22.764457+0000", 2023-07-29T13:29:44.629 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "gid": 4481, 2023-07-29T13:29:44.629 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "addr": "172.21.15.40:0/924511428", 2023-07-29T13:29:44.629 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "metadata": { 2023-07-29T13:29:44.629 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "arch": "x86_64", 2023-07-29T13:29:44.629 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "ceph_release": "reef", 2023-07-29T13:29:44.629 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "ceph_version": "ceph version 18.1.3-11-g0484097b (0484097b0572e96f42cf6402cbe0ac8dcb046577) reef (rc)", 2023-07-29T13:29:44.630 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "ceph_version_short": "18.1.3-11-g0484097b", 2023-07-29T13:29:44.630 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2023-07-29T13:29:45.128 INFO:tasks.workunit.client.0.smithi040.stdout: api_service43.290+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b54a6760 2023-07-29T13:29:45.128 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 42 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:45.128 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:45.128 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.a7bbaa1 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:45.128 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:45.129 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid noexist '@7' '@7' [read 0~256] tid 4 osd.2 2023-07-29T13:29:45.129 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:45.129 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 4 2023-07-29T13:29:45.129 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 15 client.4402.objecter _send_op 4 to 7.1 on osd.2 2023-07-29T13:29:45.129 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:4 7.1 7:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x5612b54a6c50 con 0x5612b54a4ec0 2023-07-29T13:29:45.129 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:29:45.129 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.290+0000 7fdba6a34780 5 client.4402.objecter 2 in flight 2023-07-29T13:29:45.130 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.824+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:45.130 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.824+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:45.130 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.824+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:45.130 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:43.879+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:29:45.130 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:44.825+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:45.131 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:44.825+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:45.131 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:44.825+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:45.131 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 4 ==== osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:45.131 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2023-07-29T13:29:45.131 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:45.131 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 3 ondisk uv 1 in 7.1 attempt 0 2023-07-29T13:29:45.131 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 5 2023-07-29T13:29:45.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 3 2023-07-29T13:29:45.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 15 client.4402.objecter _finish_op 3 2023-07-29T13:29:45.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:29:45.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 3 2023-07-29T13:29:45.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 5 client.4402.objecter 1 in flight 2023-07-29T13:29:45.132 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 3 ==== osd_map(43..45 src has 1..45) v4 ==== 1543+0+0 (crc 0 0 0) 0x7fdb9c0123f0 con 0x5612b54a4ec0 2023-07-29T13:29:45.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(43..45 src has 1..45) v4 2023-07-29T13:29:45.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [43,45] > 42 2023-07-29T13:29:45.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 43 2023-07-29T13:29:45.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 5 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:45.133 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-07-29T13:29:45.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:45.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 10 client.4402.objecter checking op 4 2023-07-29T13:29:45.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 43 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:45.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:45.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 44 2023-07-29T13:29:45.134 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 10 client.4402.objecter checking op 4 2023-07-29T13:29:45.135 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 44 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:45.135 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:45.135 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 45 2023-07-29T13:29:45.135 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 10 client.4402.objecter checking op 4 2023-07-29T13:29:45.135 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 45 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:45.136 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:45.136 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:45.136 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba15f8640 20 client.4402.objecter 4 7.a7bbaa1 osd.2 noexist [read 0~256] 2023-07-29T13:29:45.136 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 4 ondisk uv 0 in 7.1 attempt 0 2023-07-29T13:29:45.136 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 10 client.4402.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-07-29T13:29:45.136 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:29:45.136 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 4 2023-07-29T13:29:45.137 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 15 client.4402.objecter _finish_op 4 2023-07-29T13:29:45.137 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:45.137 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 4 2023-07-29T13:29:45.137 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:29:45.137 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (1835 ms) 2023-07-29T13:29:45.137 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2023-07-29T13:29:45.683 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b54a6760 2023-07-29T13:29:45.684 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 45 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:45.684 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:45.684 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:45.684 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:45.684 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [read 0~256] tid 5 osd.2 2023-07-29T13:29:45.684 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:45.684 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 5 2023-07-29T13:29:45.685 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 15 client.4402.objecter _send_op 5 to 7.1 on osd.2 2023-07-29T13:29:45.685 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:5 7.1 7:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45) v8 -- 0x5612b54a5c10 con 0x5612b54a4ec0 2023-07-29T13:29:45.685 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:45.685 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:29:45.685 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5561b10 2023-07-29T13:29:45.685 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 45 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:45.686 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:45.686 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.a7bbaa1 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:45.686 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:45.686 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid noexist '@7' '@7' [read 0~256] tid 6 osd.2 2023-07-29T13:29:45.686 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:45.686 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 6 2023-07-29T13:29:45.687 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 15 client.4402.objecter _send_op 6 to 7.1 on osd.2 2023-07-29T13:29:45.687 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:6 7.1 7:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45) v8 -- 0x5612b5562000 con 0x5612b54a4ec0 2023-07-29T13:29:45.687 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:29:45.687 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.126+0000 7fdba6a34780 5 client.4402.objecter 2 in flight 2023-07-29T13:29:45.688 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 6 ==== osd_map(46..46 src has 1..46) v4 ==== 320+0+0 (crc 0 0 0) 0x7fdb9c002c20 con 0x5612b54a4ec0 2023-07-29T13:29:45.688 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 7 ==== osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:45.688 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(46..46 src has 1..46) v4 2023-07-29T13:29:45.689 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2023-07-29T13:29:45.689 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:45.689 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [46,46] > 45 2023-07-29T13:29:45.689 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 46 2023-07-29T13:29:45.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 10 client.4402.objecter checking op 5 2023-07-29T13:29:45.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 46 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:45.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:45.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 10 client.4402.objecter checking op 6 2023-07-29T13:29:45.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 46 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:45.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:45.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:45.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 20 client.4402.objecter 5 7.d3a59201 osd.2 exist [read 0~256] 2023-07-29T13:29:45.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba15f8640 20 client.4402.objecter 6 7.a7bbaa1 osd.2 noexist [read 0~256] 2023-07-29T13:29:45.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 5 ondisk uv 1 in 7.1 attempt 0 2023-07-29T13:29:45.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 5 2023-07-29T13:29:45.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 5 2023-07-29T13:29:45.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 15 client.4402.objecter _finish_op 5 2023-07-29T13:29:45.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:29:45.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 5 2023-07-29T13:29:45.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 5 client.4402.objecter 1 in flight 2023-07-29T13:29:45.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 8 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:45.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-07-29T13:29:45.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:45.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 6 ondisk uv 0 in 7.1 attempt 0 2023-07-29T13:29:45.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 10 client.4402.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-07-29T13:29:45.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:29:45.696 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 6 2023-07-29T13:29:45.696 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 15 client.4402.objecter _finish_op 6 2023-07-29T13:29:45.826 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:45.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 6 2023-07-29T13:29:45.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:29:45.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (556 ms) 2023-07-29T13:29:45.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2023-07-29T13:29:45.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5561b10 2023-07-29T13:29:45.827 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 46 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:45.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:45.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:45.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:45.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [read 0~256] tid 7 osd.2 2023-07-29T13:29:45.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:45.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 7 2023-07-29T13:29:45.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba6a34780 15 client.4402.objecter _send_op 7 to 7.1 on osd.2 2023-07-29T13:29:45.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.681+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:7 7.1 7:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46) v8 -- 0x5612b54a6760 con 0x5612b54a4ec0 2023-07-29T13:29:45.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:45.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:29:45.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b55a2a40 2023-07-29T13:29:45.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 46 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:45.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:45.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.a7bbaa1 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:45.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:45.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid noexist '@7' '@7' [read 0~256] tid 8 osd.2 2023-07-29T13:29:45.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:45.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 8 2023-07-29T13:29:45.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 15 client.4402.objecter _send_op 8 to 7.1 on osd.2 2023-07-29T13:29:45.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:8 7.1 7:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46) v8 -- 0x5612b55a2f30 con 0x5612b54a4ec0 2023-07-29T13:29:45.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:29:45.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.682+0000 7fdba6a34780 5 client.4402.objecter 2 in flight 2023-07-29T13:29:45.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.825+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:45.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.825+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:45.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.825+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:45.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.861+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 9 ==== osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:45.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.861+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2023-07-29T13:29:45.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.861+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:45.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.861+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 7 ondisk uv 1 in 7.1 attempt 0 2023-07-29T13:29:45.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.861+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 5 2023-07-29T13:29:45.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.861+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 7 2023-07-29T13:29:45.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.861+0000 7fdba45fe640 15 client.4402.objecter _finish_op 7 2023-07-29T13:29:45.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.861+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:29:45.874 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.861+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 7 2023-07-29T13:29:45.874 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.861+0000 7fdba45fe640 5 client.4402.objecter 1 in flight 2023-07-29T13:29:45.874 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 10 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:45.875 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-07-29T13:29:45.875 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:45.875 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 8 ondisk uv 0 in 7.1 attempt 0 2023-07-29T13:29:45.875 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba45fe640 10 client.4402.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-07-29T13:29:45.876 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:29:45.876 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 8 2023-07-29T13:29:45.876 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba45fe640 15 client.4402.objecter _finish_op 8 2023-07-29T13:29:45.877 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:45.877 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 8 2023-07-29T13:29:45.877 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:29:45.877 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (187 ms) 2023-07-29T13:29:45.878 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2023-07-29T13:29:45.878 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2023-07-29T13:29:45.878 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 20 librados: queue_aio_write 0x5612b538edf0 completion 0x5612b54a6b10 write_seq 1 2023-07-29T13:29:45.878 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5561b10 2023-07-29T13:29:45.878 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 46 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:46.225 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:02:41.559 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:46.234 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:02:41.568 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:46.235 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:02:41.569 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:46.302 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:02:41.586 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:46.304 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:02:41.588 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:46.311 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:02:41.645 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:46.392 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:02:41.726 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:46.406 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:02:41.690 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:46.454 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:02:41.788 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:29:47.434 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:47.434 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:47.434 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:47.434 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [write 0~5 in=5b] tid 9 osd.2 2023-07-29T13:29:47.435 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:47.435 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 9 2023-07-29T13:29:47.435 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 15 client.4402.objecter _send_op 9 to 7.1 on osd.2 2023-07-29T13:29:47.435 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:9 7.1 7:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46) v8 -- 0x5612b55633c0 con 0x5612b54a4ec0 2023-07-29T13:29:47.435 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:47.435 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:29:47.435 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:45.868+0000 7fdba6a34780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2023-07-29T13:29:47.436 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:46.825+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:47.436 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:46.825+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:47.436 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:46.825+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:47.436 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:46.825+0000 7fdb8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:29:47.436 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 11 ==== osd_op_reply(9 exist [write 0~5] v46'2 uv2 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:47.436 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(9 exist [write 0~5] v46'2 uv2 ondisk = 0) v8 2023-07-29T13:29:47.436 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:47.437 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 9 ondisk uv 2 in 7.1 attempt 0 2023-07-29T13:29:47.437 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:29:47.437 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 9 2023-07-29T13:29:47.437 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba45fe640 15 client.4402.objecter _finish_op 9 2023-07-29T13:29:47.437 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:47.437 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 9 2023-07-29T13:29:47.437 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:29:47.437 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba45fe640 20 librados: complete_aio_write 0x5612b54a6b10 2023-07-29T13:29:47.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (1563 ms) 2023-07-29T13:29:47.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2023-07-29T13:29:47.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2023-07-29T13:29:47.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 20 librados: queue_aio_write 0x5612b538edf0 completion 0x5612b54a0110 write_seq 2 2023-07-29T13:29:47.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5561b10 2023-07-29T13:29:48.844 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 46 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:48.844 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:48.844 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:48.844 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:48.844 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [write 0~5 in=5b] tid 10 osd.2 2023-07-29T13:29:48.845 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:48.845 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 10 2023-07-29T13:29:48.845 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 15 client.4402.objecter _send_op 10 to 7.1 on osd.2 2023-07-29T13:29:48.845 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:10 7.1 7:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46) v8 -- 0x5612b55633c0 con 0x5612b54a4ec0 2023-07-29T13:29:48.845 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:48.845 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:29:48.845 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.432+0000 7fdba6a34780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2023-07-29T13:29:48.845 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.611+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 12 ==== osd_map(47..47 src has 1..48) v4 ==== 2422+0+0 (crc 0 0 0) 0x7fdb9c002a80 con 0x5612b54a4ec0 2023-07-29T13:29:48.846 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.611+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(47..47 src has 1..48) v4 2023-07-29T13:29:48.846 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.611+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [47,47] > 46 2023-07-29T13:29:48.846 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.611+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 47 2023-07-29T13:29:48.846 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.611+0000 7fdba15f8640 10 client.4402.objecter checking op 10 2023-07-29T13:29:48.846 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.611+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 47 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:48.846 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.611+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:48.846 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.611+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:48.847 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.611+0000 7fdba15f8640 20 client.4402.objecter 10 7.d3a59201 osd.2 exist [write 0~5 in=5b] 2023-07-29T13:29:48.847 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.825+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:48.847 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.825+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:48.847 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:47.825+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:48.847 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.825+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:48.847 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.825+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:48.847 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.825+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:48.848 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 13 ==== osd_op_reply(10 exist [write 0~5] v47'3 uv3 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:50.057 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:29:50.014+0000 12a61640 -1 osd.7 50 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4477.0:2 24.1a 24:5c6776db:::0.obj:head [write 0~94 in=94b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-07-29T13:29:50.413 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(10 exist [write 0~5] v47'3 uv3 ondisk = 0) v8 2023-07-29T13:29:50.413 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:50.413 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 10 ondisk uv 3 in 7.1 attempt 0 2023-07-29T13:29:50.413 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:29:50.414 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 10 2023-07-29T13:29:50.414 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba45fe640 15 client.4402.objecter _finish_op 10 2023-07-29T13:29:50.414 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:50.414 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 10 2023-07-29T13:29:50.414 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:29:50.414 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba45fe640 20 librados: complete_aio_write 0x5612b54a0110 2023-07-29T13:29:50.414 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (1410 ms) 2023-07-29T13:29:50.415 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2023-07-29T13:29:50.415 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2023-07-29T13:29:50.415 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 20 librados: queue_aio_write 0x5612b538edf0 completion 0x5612b5561b10 write_seq 3 2023-07-29T13:29:50.415 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b55857e0 2023-07-29T13:29:50.416 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 47 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:50.416 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:50.416 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:50.416 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:50.416 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [write 0~5 in=5b] tid 11 osd.2 2023-07-29T13:29:50.416 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:50.417 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 11 2023-07-29T13:29:50.417 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 15 client.4402.objecter _send_op 11 to 7.1 on osd.2 2023-07-29T13:29:50.417 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:11 7.1 7:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x5612b5585cd0 con 0x5612b54a4ec0 2023-07-29T13:29:50.417 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:50.417 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:29:50.417 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.842+0000 7fdba6a34780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2023-07-29T13:29:50.418 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:48.880+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:29:50.418 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:49.825+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:50.418 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:49.825+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:50.418 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:49.825+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:52.482 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 14 ==== osd_map(48..49 src has 1..49) v4 ==== 4232+0+0 (crc 0 0 0) 0x7fdb9c002040 con 0x5612b54a4ec0 2023-07-29T13:29:52.482 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(48..49 src has 1..49) v4 2023-07-29T13:29:52.482 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [48,49] > 47 2023-07-29T13:29:52.482 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 48 2023-07-29T13:29:52.482 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 10 client.4402.objecter checking op 11 2023-07-29T13:29:52.483 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 48 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:52.483 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:52.483 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 49 2023-07-29T13:29:52.483 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 10 client.4402.objecter checking op 11 2023-07-29T13:29:52.483 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 49 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:29:52.483 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:52.483 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:52.483 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.411+0000 7fdba15f8640 20 client.4402.objecter 11 7.d3a59201 osd.2 exist [write 0~5 in=5b] 2023-07-29T13:29:52.484 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.825+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:52.484 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.825+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:52.484 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:50.825+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:52.484 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:51.825+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:52.484 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:51.825+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:52.484 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:51.825+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:52.484 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 15 ==== osd_op_reply(11 exist [write 0~5] v49'4 uv4 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:52.485 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(11 exist [write 0~5] v49'4 uv4 ondisk = 0) v8 2023-07-29T13:29:52.485 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:52.485 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 11 ondisk uv 4 in 7.1 attempt 0 2023-07-29T13:29:52.485 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:29:52.485 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 11 2023-07-29T13:29:52.485 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba45fe640 15 client.4402.objecter _finish_op 11 2023-07-29T13:29:52.485 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:52.486 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 11 2023-07-29T13:29:52.486 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:29:52.486 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba45fe640 20 librados: complete_aio_write 0x5612b5561b10 2023-07-29T13:29:52.486 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (3638 ms) 2023-07-29T13:29:52.486 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2023-07-29T13:29:52.575 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:29:52.560+0000 12a61640 -1 osd.7 52 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4477.0:2 24.1a 24:5c6776db:::0.obj:head [write 0~94 in=94b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-07-29T13:29:53.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07 7f9a0b838780 1 -- 172.21.15.40:0/207577766 shutdown_connections 2023-07-29T13:29:53.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 1 --2- 172.21.15.40:0/207577766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557ec1a98ae0 0x557ec1a9ba30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:53.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 1 --2- 172.21.15.40:0/207577766 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x557ec1a05a00 0x557ec1a07270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:53.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 1 --2- 172.21.15.40:0/207577766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ec1a03ca0 0x557ec1a99190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:29:53.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 1 -- 172.21.15.40:0/207577766 >> 172.21.15.40:0/207577766 conn(0x557ec19be570 msgr2=0x557ec1a96920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:29:53.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 1 -- 172.21.15.40:0/207577766 shutdown_connections 2023-07-29T13:29:53.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 1 -- 172.21.15.40:0/207577766 wait complete. 2023-07-29T13:29:53.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.088+0000 7f9a0b838780 1 librados: shutdown 2023-07-29T13:29:53.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 1 2023-07-29T13:29:53.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2023-07-29T13:29:53.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.096+0000 7f9a0b838780 -1 failed for service _ceph-mon._tcp 2023-07-29T13:29:53.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:14.096+0000 7f9a0b838780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2023-07-29T13:29:53.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (390 ms) 2023-07-29T13:29:53.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [----------] 1 test from LibRadosMiscConnectFailure (390 ms total) 2023-07-29T13:29:53.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:29:53.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2023-07-29T13:29:53.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2023-07-29T13:29:53.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec108a110 2023-07-29T13:29:53.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec19ac630 2023-07-29T13:29:53.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec0fefbc0 2023-07-29T13:29:53.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1a00360 2023-07-29T13:29:53.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec0db1710 2023-07-29T13:29:53.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1a06ca0 2023-07-29T13:29:53.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec19c8ad0 2023-07-29T13:29:53.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1128a40 2023-07-29T13:29:53.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1129540 2023-07-29T13:29:53.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112a0d0 2023-07-29T13:29:53.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112ac40 2023-07-29T13:29:53.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112b370 2023-07-29T13:29:53.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112a880 2023-07-29T13:29:53.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112aa10 2023-07-29T13:29:53.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1aa96a0 2023-07-29T13:29:53.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1129d60 2023-07-29T13:29:53.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1aaa230 2023-07-29T13:29:53.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1aaad10 2023-07-29T13:29:53.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec11292f0 2023-07-29T13:29:53.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1aabd10 2023-07-29T13:29:53.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1aaaa40 2023-07-29T13:29:53.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1aac8d0 2023-07-29T13:29:53.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112b9d0 2023-07-29T13:29:53.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112c0f0 2023-07-29T13:29:53.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112cbd0 2023-07-29T13:29:53.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1aaba10 2023-07-29T13:29:53.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112d790 2023-07-29T13:29:53.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112e280 2023-07-29T13:29:53.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112c9b0 2023-07-29T13:29:53.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112f260 2023-07-29T13:29:53.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112e020 2023-07-29T13:29:53.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec11301d0 2023-07-29T13:29:53.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec112ef50 2023-07-29T13:29:53.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1130de0 2023-07-29T13:29:53.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 0x557ec1131940 2023-07-29T13:29:53.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: started 35 aios 2023-07-29T13:29:53.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec108a110 2023-07-29T13:29:53.194 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec19ac630 2023-07-29T13:29:53.194 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec0fefbc0 2023-07-29T13:29:53.827 INFO:tasks.workunit.client.0.smithi040.stdout: -29T13:29:52.480+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5562600 2023-07-29T13:29:53.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 49 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:53.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:53.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:53.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:53.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [read 0~0] tid 12 osd.2 2023-07-29T13:29:53.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:53.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 12 2023-07-29T13:29:53.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 15 client.4402.objecter _send_op 12 to 7.1 on osd.2 2023-07-29T13:29:53.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:12 7.1 7:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49) v8 -- 0x5612b5562af0 con 0x5612b54a4ec0 2023-07-29T13:29:53.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:53.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:29:53.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5565160 2023-07-29T13:29:53.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 49 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:53.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:53.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.a7bbaa1 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:53.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:53.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid noexist '@7' '@7' [read 0~0] tid 13 osd.2 2023-07-29T13:29:53.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:53.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 13 2023-07-29T13:29:53.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 15 client.4402.objecter _send_op 13 to 7.1 on osd.2 2023-07-29T13:29:53.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:13 7.1 7:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49) v8 -- 0x5612b5562fb0 con 0x5612b54a4ec0 2023-07-29T13:29:53.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:29:53.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.480+0000 7fdba6a34780 5 client.4402.objecter 2 in flight 2023-07-29T13:29:53.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.825+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:53.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.825+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:53.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:52.825+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:53.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:53.825+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:53.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:53.825+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:54.022 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:29:53.894+0000 12a61640 -1 osd.7 53 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4477.0:2 24.1a 24:5c6776db:::0.obj:head [write 0~94 in=94b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-07-29T13:29:54.910 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:53.825+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:54.910 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:53.881+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:29:54.910 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.825+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:54.910 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.825+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:54.911 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.825+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:54.911 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.907+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 16 ==== osd_map(50..52 src has 1..52) v4 ==== 3797+0+0 (crc 0 0 0) 0x7fdb9c002040 con 0x5612b54a4ec0 2023-07-29T13:29:54.911 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.907+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 17 ==== osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:54.911 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.907+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(50..52 src has 1..52) v4 2023-07-29T13:29:54.911 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.907+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2023-07-29T13:29:54.911 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.907+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:54.912 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.907+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [50,52] > 49 2023-07-29T13:29:54.912 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.907+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 50 2023-07-29T13:29:54.912 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 10 client.4402.objecter checking op 12 2023-07-29T13:29:54.912 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 50 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:54.912 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:54.912 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 10 client.4402.objecter checking op 13 2023-07-29T13:29:54.912 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 50 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:54.912 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:54.913 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 51 2023-07-29T13:29:54.913 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 10 client.4402.objecter checking op 12 2023-07-29T13:29:54.913 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 51 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:54.913 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:54.913 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 10 client.4402.objecter checking op 13 2023-07-29T13:29:54.913 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 51 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:54.913 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:54.914 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 52 2023-07-29T13:29:54.914 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 10 client.4402.objecter checking op 12 2023-07-29T13:29:54.914 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 52 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:54.914 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:54.914 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 10 client.4402.objecter checking op 13 2023-07-29T13:29:54.914 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 52 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:54.914 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:54.914 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:54.915 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter 12 7.d3a59201 osd.2 exist [read 0~0] 2023-07-29T13:29:54.915 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba15f8640 20 client.4402.objecter 13 7.a7bbaa1 osd.2 noexist [read 0~0] 2023-07-29T13:29:54.915 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 12 ondisk uv 4 in 7.1 attempt 0 2023-07-29T13:29:54.915 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 5 2023-07-29T13:29:54.915 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 12 2023-07-29T13:29:54.915 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 15 client.4402.objecter _finish_op 12 2023-07-29T13:29:54.915 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:29:54.916 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 12 2023-07-29T13:29:54.916 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 5 client.4402.objecter 1 in flight 2023-07-29T13:29:54.916 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 18 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:54.916 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-07-29T13:29:54.916 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:54.916 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 13 ondisk uv 0 in 7.1 attempt 0 2023-07-29T13:29:54.916 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 10 client.4402.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-07-29T13:29:54.916 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:29:54.917 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 13 2023-07-29T13:29:54.917 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 15 client.4402.objecter _finish_op 13 2023-07-29T13:29:54.917 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:54.917 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 13 2023-07-29T13:29:54.917 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:29:54.917 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (2428 ms) 2023-07-29T13:29:54.917 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2023-07-29T13:29:54.917 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5562600 2023-07-29T13:29:54.918 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 52 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:54.918 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:54.918 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:55.203 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.90_pp: "distro": "centos", 2023-07-29T13:29:55.203 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2023-07-29T13:29:55.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "distro_version": "9", 2023-07-29T13:29:55.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "foo": "bar", 2023-07-29T13:29:55.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "hostname": "smithi040", 2023-07-29T13:29:55.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Mon Jul 24 09:26:29 UTC 2023", 2023-07-29T13:29:55.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "kernel_version": "5.14.0-344.el9.x86_64", 2023-07-29T13:29:55.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "mem_swap_kb": "0", 2023-07-29T13:29:55.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "mem_total_kb": "32495004", 2023-07-29T13:29:55.204 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "os": "Linux", 2023-07-29T13:29:55.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "this": "that" 2023-07-29T13:29:55.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: }, 2023-07-29T13:29:55.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "task_status": {} 2023-07-29T13:29:55.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: } 2023-07-29T13:29:55.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: } 2023-07-29T13:29:55.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: } 2023-07-29T13:29:55.205 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: } 2023-07-29T13:29:55.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: } 2023-07-29T13:29:55.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: 2023-07-29T13:29:55.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: attempt 1 of 20 2023-07-29T13:29:55.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: failed to deregister: 2023-07-29T13:29:55.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: { 2023-07-29T13:29:55.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "epoch": 12, 2023-07-29T13:29:55.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "modified": "2023-07-29T13:29:45.041229+0000", 2023-07-29T13:29:55.206 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "services": { 2023-07-29T13:29:55.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "laundry": { 2023-07-29T13:29:55.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "daemons": { 2023-07-29T13:29:55.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "summary": "", 2023-07-29T13:29:55.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "close-test-pid91371": { 2023-07-29T13:29:55.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "start_epoch": 12, 2023-07-29T13:29:55.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "start_stamp": "2023-07-29T13:29:44.706837+0000", 2023-07-29T13:29:55.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "gid": 4654, 2023-07-29T13:29:55.207 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "addr": "172.21.15.40:0/219478056", 2023-07-29T13:29:55.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "metadata": { 2023-07-29T13:29:55.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "arch": "x86_64", 2023-07-29T13:29:55.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "ceph_release": "reef", 2023-07-29T13:29:55.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "ceph_version": "ceph version 18.1.3-11-g0484097b (0484097b0572e96f42cf6402cbe0ac8dcb046577) reef (rc)", 2023-07-29T13:29:55.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "ceph_version_short": "18.1.3-11-g0484097b", 2023-07-29T13:29:55.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2023-07-29T13:29:55.208 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "distro": "centos", 2023-07-29T13:29:55.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2023-07-29T13:29:55.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "distro_version": "9", 2023-07-29T13:29:55.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "foo": "bar", 2023-07-29T13:29:55.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "hostname": "smithi040", 2023-07-29T13:29:55.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Mon Jul 24 09:26:29 UTC 2023", 2023-07-29T13:29:55.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "kernel_version": "5.14.0-344.el9.x86_64", 2023-07-29T13:29:55.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "mem_swap_kb": "0", 2023-07-29T13:29:55.209 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "mem_total_kb": "32495004", 2023-07-29T13:29:55.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "os": "Linux", 2023-07-29T13:29:55.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "this": "that" 2023-07-29T13:29:55.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: }, 2023-07-29T13:29:55.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: "task_status": {} 2023-07-29T13:29:55.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: } 2023-07-29T13:29:55.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: } 2023-07-29T13:29:55.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: } 2023-07-29T13:29:55.210 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: } 2023-07-29T13:29:55.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: } 2023-07-29T13:29:55.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: 2023-07-29T13:29:55.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: attempt 2 of 20 2023-07-29T13:29:55.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (15762 ms) 2023-07-29T13:29:55.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (41296 ms total) 2023-07-29T13:29:55.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: 2023-07-29T13:29:55.211 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [----------] Global test environment tear-down 2023-07-29T13:29:55.212 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (41296 ms total) 2023-07-29T13:29:55.212 INFO:tasks.workunit.client.0.smithi040.stdout: api_service_pp: [ PASSED ] 4 tests. 2023-07-29T13:29:58.554 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:29:58.543+0000 12a61640 -1 osd.6 56 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4458.0:2 16.10 16:0afc9024:test-rados-api-smithi040-91392-3::testobj:head [write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) 2023-07-29T13:29:58.828 INFO:tasks.workunit.client.0.smithi040.stdout:8+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:58.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [read 0~0] tid 14 osd.2 2023-07-29T13:29:58.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:58.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 14 2023-07-29T13:29:58.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 15 client.4402.objecter _send_op 14 to 7.1 on osd.2 2023-07-29T13:29:58.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:14 7.1 7:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52) v8 -- 0x5612b5562af0 con 0x5612b54a4ec0 2023-07-29T13:29:58.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:58.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:29:58.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5565160 2023-07-29T13:29:58.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 52 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:58.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:58.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.a7bbaa1 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:58.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:58.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid noexist '@7' '@7' [read 0~0] tid 15 osd.2 2023-07-29T13:29:58.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:58.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 15 2023-07-29T13:29:58.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 15 client.4402.objecter _send_op 15 to 7.1 on osd.2 2023-07-29T13:29:58.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:15 7.1 7:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52) v8 -- 0x5612b5565650 con 0x5612b54a4ec0 2023-07-29T13:29:58.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:29:58.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:54.908+0000 7fdba6a34780 5 client.4402.objecter 2 in flight 2023-07-29T13:29:58.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:55.826+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:58.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:55.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:58.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:55.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:58.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:56.826+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:58.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:56.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:58.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:56.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:58.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:56.826+0000 7fdb8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:29:58.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:57.826+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:58.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:57.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:58.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:57.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:58.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.826+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:29:58.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:29:58.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:29:58.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 20 ==== osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:58.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 19 ==== osd_map(53..54 src has 1..55) v4 ==== 2722+0+0 (crc 0 0 0) 0x7fdb9c002040 con 0x5612b54a4ec0 2023-07-29T13:29:58.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2023-07-29T13:29:58.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:58.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 14 ondisk uv 4 in 7.1 attempt 0 2023-07-29T13:29:58.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(53..54 src has 1..55) v4 2023-07-29T13:29:58.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 5 2023-07-29T13:29:58.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 14 2023-07-29T13:29:58.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 15 client.4402.objecter _finish_op 14 2023-07-29T13:29:58.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:29:58.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 14 2023-07-29T13:29:58.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 5 client.4402.objecter 1 in flight 2023-07-29T13:29:58.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [53,54] > 52 2023-07-29T13:29:58.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 53 2023-07-29T13:29:58.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 21 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:29:58.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-07-29T13:29:58.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:29:58.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 10 client.4402.objecter checking op 15 2023-07-29T13:29:58.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 53 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:58.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:58.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 54 2023-07-29T13:29:58.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 10 client.4402.objecter checking op 15 2023-07-29T13:29:58.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 54 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:58.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:58.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:29:58.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba15f8640 20 client.4402.objecter 15 7.a7bbaa1 osd.2 noexist [read 0~0] 2023-07-29T13:29:58.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 15 ondisk uv 0 in 7.1 attempt 0 2023-07-29T13:29:58.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 10 client.4402.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-07-29T13:29:58.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:29:58.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 15 2023-07-29T13:29:58.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 15 client.4402.objecter _finish_op 15 2023-07-29T13:29:58.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:58.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 15 2023-07-29T13:29:58.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:29:58.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (3952 ms) 2023-07-29T13:29:58.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2023-07-29T13:29:58.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5585170 2023-07-29T13:29:58.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 54 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:58.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:29:58.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:58.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:58.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [read 0~0] tid 16 osd.2 2023-07-29T13:29:58.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:29:58.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 16 2023-07-29T13:29:58.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 15 client.4402.objecter _send_op 16 to 7.1 on osd.2 2023-07-29T13:29:58.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:16 7.1 7:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54) v8 -- 0x5612b5585660 con 0x5612b54a4ec0 2023-07-29T13:29:58.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:58.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:29:58.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b55a5b30 2023-07-29T13:29:58.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 54 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:29:58.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:29:58.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.a7bbaa1 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:29:58.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:58.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid noexist '@7' '@7' [read 0~0] tid 17 osd.2 2023-07-29T13:29:58.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:29:59.785 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:29:59.716+0000 12a61640 -1 osd.6 57 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4458.0:2 16.10 16:0afc9024:test-rados-api-smithi040-91392-3::testobj:head [write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) 2023-07-29T13:30:00.931 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:30:00.808+0000 12a61640 -1 osd.6 58 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4458.0:2 16.10 16:0afc9024:test-rados-api-smithi040-91392-3::testobj:head [write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) 2023-07-29T13:30:01.903 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:30:01.875+0000 12a61640 -1 osd.6 59 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4458.0:2 16.10 16:0afc9024:test-rados-api-smithi040-91392-3::testobj:head [write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) 2023-07-29T13:30:02.977 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:30:02.946+0000 12a61640 -1 osd.6 61 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4458.0:2 16.10 16:0afc9024:test-rados-api-smithi040-91392-3::testobj:head [write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) 2023-07-29T13:30:04.390 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:30:04.380+0000 12a61640 -1 osd.6 62 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4458.0:2 16.10 16:0afc9024:test-rados-api-smithi040-91392-3::testobj:head [write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) 2023-07-29T13:30:05.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 17 2023-07-29T13:30:05.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 15 client.4402.objecter _send_op 17 to 7.1 on osd.2 2023-07-29T13:30:05.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:17 7.1 7:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54) v8 -- 0x5612b55a6020 con 0x5612b54a4ec0 2023-07-29T13:30:05.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:30:05.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.860+0000 7fdba6a34780 5 client.4402.objecter 2 in flight 2023-07-29T13:30:05.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:58.882+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:30:05.186 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:59.826+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:05.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:59.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:05.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:29:59.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:05.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:00.826+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:05.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:00.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:05.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:00.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:05.187 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:01.826+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:05.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:01.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:05.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:01.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:05.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:02.826+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:05.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:02.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:05.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:02.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:05.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:03.826+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:05.188 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:03.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:05.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:03.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:05.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:03.883+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:30:05.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:04.826+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:05.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:04.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:05.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:04.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:05.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.183+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 22 ==== osd_map(55..57 src has 1..59) v4 ==== 5444+0+0 (crc 0 0 0) 0x7fdb9c002040 con 0x5612b54a4ec0 2023-07-29T13:30:05.189 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.183+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 23 ==== osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:30:05.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(55..57 src has 1..59) v4 2023-07-29T13:30:05.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2023-07-29T13:30:05.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:30:05.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [55,57] > 54 2023-07-29T13:30:05.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 55 2023-07-29T13:30:05.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 10 client.4402.objecter checking op 16 2023-07-29T13:30:05.190 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 55 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:30:05.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:05.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 10 client.4402.objecter checking op 17 2023-07-29T13:30:05.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 55 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:30:05.191 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:30:05.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 56 2023-07-29T13:30:05.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 10 client.4402.objecter checking op 16 2023-07-29T13:30:05.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 56 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:30:05.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:05.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 10 client.4402.objecter checking op 17 2023-07-29T13:30:05.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 56 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:30:05.192 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:30:05.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 57 2023-07-29T13:30:05.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 10 client.4402.objecter checking op 16 2023-07-29T13:30:05.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 57 base exist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:30:05.193 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:05.194 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 10 client.4402.objecter checking op 17 2023-07-29T13:30:05.194 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 57 base noexist @7 precalc_pgid 0 pgid 0.0 is_read 2023-07-29T13:30:05.194 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter _calc_target target noexist @7 -> pgid 7.a7bbaa1 2023-07-29T13:30:05.195 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:30:05.195 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter 16 7.d3a59201 osd.2 exist [read 0~0] 2023-07-29T13:30:05.195 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba15f8640 20 client.4402.objecter 17 7.a7bbaa1 osd.2 noexist [read 0~0] 2023-07-29T13:30:05.195 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 16 ondisk uv 4 in 7.1 attempt 0 2023-07-29T13:30:05.196 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 5 2023-07-29T13:30:05.197 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 16 2023-07-29T13:30:05.197 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 15 client.4402.objecter _finish_op 16 2023-07-29T13:30:05.197 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 5 2023-07-29T13:30:05.197 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 16 2023-07-29T13:30:05.198 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 5 client.4402.objecter 1 in flight 2023-07-29T13:30:05.579 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:30:05.567+0000 12a61640 -1 osd.6 64 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4458.0:2 16.10 16:0afc9024:test-rados-api-smithi040-91392-3::testobj:head [write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) 2023-07-29T13:30:06.380 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:03:01.714 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:06.383 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:03:01.717 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:06.458 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:03:01.742 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:06.462 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:03:01.746 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:06.464 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:03:01.799 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:06.467 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:03:01.801 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:06.519 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:03:01.853 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:06.574 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:03:01.908 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:06.585 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:03:01.870 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:06.665 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:30:06.646+0000 12a61640 -1 osd.6 65 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4458.0:2 16.10 16:0afc9024:test-rados-api-smithi040-91392-3::testobj:head [write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) 2023-07-29T13:30:06.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 24 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:30:06.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-07-29T13:30:06.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:30:06.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 17 ondisk uv 0 in 7.1 attempt 0 2023-07-29T13:30:06.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 10 client.4402.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-07-29T13:30:06.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:30:06.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 17 2023-07-29T13:30:06.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 15 client.4402.objecter _finish_op 17 2023-07-29T13:30:06.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:30:06.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 17 2023-07-29T13:30:06.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:30:06.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (6324 ms) 2023-07-29T13:30:06.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2023-07-29T13:30:06.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 20 librados: queue_aio_write 0x5612b538edf0 completion 0x5612b5562600 write_seq 4 2023-07-29T13:30:06.833 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5562710 2023-07-29T13:30:06.833 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 57 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:06.833 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:06.834 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:30:06.834 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:30:06.834 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [writefull 0~5 in=5b] tid 18 osd.2 2023-07-29T13:30:06.835 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:30:06.835 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 18 2023-07-29T13:30:06.835 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 15 client.4402.objecter _send_op 18 to 7.1 on osd.2 2023-07-29T13:30:06.835 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:18 7.1 7:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57) v8 -- 0x5612b55a5b30 con 0x5612b54a4ec0 2023-07-29T13:30:06.835 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:30:06.835 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.184+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:30:06.835 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.826+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:06.836 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:06.836 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:05.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:07.278 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:06.8 api_pool: Running main() from gmock_main.cc 2023-07-29T13:30:07.278 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2023-07-29T13:30:07.278 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [----------] Global test environment set-up. 2023-07-29T13:30:07.279 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [----------] 9 tests from LibRadosPools 2023-07-29T13:30:07.279 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2023-07-29T13:30:07.280 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ OK ] LibRadosPools.PoolList (7100 ms) 2023-07-29T13:30:07.280 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2023-07-29T13:30:07.280 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3925 ms) 2023-07-29T13:30:07.280 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2023-07-29T13:30:07.281 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4221 ms) 2023-07-29T13:30:07.281 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2023-07-29T13:30:07.281 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3915 ms) 2023-07-29T13:30:07.282 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2023-07-29T13:30:07.282 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (4953 ms) 2023-07-29T13:30:07.282 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2023-07-29T13:30:07.282 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (9426 ms) 2023-07-29T13:30:07.283 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2023-07-29T13:30:07.283 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5747 ms) 2023-07-29T13:30:07.283 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2023-07-29T13:30:07.283 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5725 ms) 2023-07-29T13:30:07.284 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2023-07-29T13:30:07.284 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8523 ms) 2023-07-29T13:30:07.284 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [----------] 9 tests from LibRadosPools (53535 ms total) 2023-07-29T13:30:07.284 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: 2023-07-29T13:30:07.285 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [----------] Global test environment tear-down 2023-07-29T13:30:07.285 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (53535 ms total) 2023-07-29T13:30:07.285 INFO:tasks.workunit.client.0.smithi040.stdout: api_pool: [ PASSED ] 9 tests. 2023-07-29T13:30:07.667 INFO:tasks.workunit.client.0.smithi040.stdout:26+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:07.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:06.826+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:07.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:06.826+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:07.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:06.827+0000 7fdb8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T13:30:07.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 25 ==== osd_map(58..63 src has 1..64) v4 ==== 4870+0+0 (crc 0 0 0) 0x7fdb9c0022d0 con 0x5612b54a4ec0 2023-07-29T13:30:07.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(58..63 src has 1..64) v4 2023-07-29T13:30:07.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [58,63] > 57 2023-07-29T13:30:07.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 58 2023-07-29T13:30:07.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 10 client.4402.objecter checking op 18 2023-07-29T13:30:07.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 58 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:07.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:07.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 59 2023-07-29T13:30:07.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 10 client.4402.objecter checking op 18 2023-07-29T13:30:07.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 59 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:07.670 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:07.670 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 60 2023-07-29T13:30:07.670 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 10 client.4402.objecter checking op 18 2023-07-29T13:30:07.670 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 60 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:07.670 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.665+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:07.670 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 61 2023-07-29T13:30:07.670 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 10 client.4402.objecter checking op 18 2023-07-29T13:30:07.670 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 61 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:07.671 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:07.671 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 62 2023-07-29T13:30:07.671 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 10 client.4402.objecter checking op 18 2023-07-29T13:30:07.671 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 62 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:07.671 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:07.671 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 63 2023-07-29T13:30:07.671 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 10 client.4402.objecter checking op 18 2023-07-29T13:30:07.672 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 63 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:07.828 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:07.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:30:07.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 10 monclient: _renew_subs 2023-07-29T13:30:07.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:30:07.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=64}) v3 -- 0x7fdb900710b0 con 0x5612b54ab210 2023-07-29T13:30:07.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:30:07.829 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.666+0000 7fdba15f8640 20 client.4402.objecter 18 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:30:07.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 12 ==== osd_map(64..66 src has 1..66) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fdb98087290 con 0x5612b54ab210 2023-07-29T13:30:07.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(64..66 src has 1..66) v4 2023-07-29T13:30:07.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [64,66] > 63 2023-07-29T13:30:07.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 64 2023-07-29T13:30:07.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 10 client.4402.objecter checking op 18 2023-07-29T13:30:07.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 64 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:07.830 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:07.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 65 2023-07-29T13:30:07.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 10 client.4402.objecter checking op 18 2023-07-29T13:30:07.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 65 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:07.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:07.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 66 2023-07-29T13:30:07.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 10 client.4402.objecter checking op 18 2023-07-29T13:30:07.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 66 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:07.831 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:07.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:30:07.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 10 monclient: _renew_subs 2023-07-29T13:30:07.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:30:07.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=67}) v3 -- 0x7fdb900710b0 con 0x5612b54ab210 2023-07-29T13:30:07.832 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:30:07.833 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.670+0000 7fdba15f8640 20 client.4402.objecter 18 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:30:08.233 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.827+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:08.234 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.827+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:08.234 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:07.827+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:08.234 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 13 ==== osd_map(67..67 src has 1..67) v4 ==== 628+0+0 (secure 0 0 0) 0x7fdb98087290 con 0x5612b54ab210 2023-07-29T13:30:08.234 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(67..67 src has 1..67) v4 2023-07-29T13:30:08.234 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [67,67] > 66 2023-07-29T13:30:08.234 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 67 2023-07-29T13:30:08.234 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 10 client.4402.objecter checking op 18 2023-07-29T13:30:08.235 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 67 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:08.235 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:08.235 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:30:08.235 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 10 monclient: _renew_subs 2023-07-29T13:30:08.235 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:30:08.235 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=68}) v3 -- 0x7fdb900710b0 con 0x5612b54ab210 2023-07-29T13:30:08.235 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:30:08.235 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.220+0000 7fdba15f8640 20 client.4402.objecter 18 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:30:08.236 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.231+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 26 ==== osd_op_reply(18 exist [writefull 0~5] v63'5 uv5 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:30:08.236 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.231+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(18 exist [writefull 0~5] v63'5 uv5 ondisk = 0) v8 2023-07-29T13:30:08.236 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.231+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:30:08.236 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.231+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 18 ondisk uv 5 in 7.1 attempt 0 2023-07-29T13:30:08.236 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.231+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:30:08.236 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.231+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 18 2023-07-29T13:30:08.236 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.231+0000 7fdba45fe640 15 client.4402.objecter _finish_op 18 2023-07-29T13:30:08.237 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.231+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:30:08.237 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.231+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 18 2023-07-29T13:30:08.237 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.231+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:30:08.237 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.231+0000 7fdba45fe640 20 librados: complete_aio_write 0x5612b5562600 2023-07-29T13:30:08.237 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (3048 ms) 2023-07-29T13:30:08.237 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2023-07-29T13:30:08.237 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 20 librados: queue_aio_write 0x5612b538edf0 completion 0x5612b55a5ee0 write_seq 5 2023-07-29T13:30:09.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5562600 2023-07-29T13:30:09.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 67 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:09.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:09.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:30:09.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:30:09.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [writefull 0~5 in=5b] tid 19 osd.2 2023-07-29T13:30:09.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:30:09.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 19 2023-07-29T13:30:09.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 15 client.4402.objecter _send_op 19 to 7.1 on osd.2 2023-07-29T13:30:09.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:19 7.1 7:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5612b55a5b30 con 0x5612b54a4ec0 2023-07-29T13:30:09.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:30:09.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.232+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:30:09.381 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.827+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:09.381 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.827+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:09.381 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.827+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:09.381 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:08.884+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:30:09.381 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 14 ==== osd_map(68..68 src has 1..68) v4 ==== 288+0+0 (secure 0 0 0) 0x7fdb98087290 con 0x5612b54ab210 2023-07-29T13:30:09.381 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(68..68 src has 1..68) v4 2023-07-29T13:30:09.381 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [68,68] > 67 2023-07-29T13:30:09.382 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 68 2023-07-29T13:30:09.382 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 10 client.4402.objecter checking op 19 2023-07-29T13:30:09.382 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 68 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:09.382 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:09.382 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:30:09.382 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 10 monclient: _renew_subs 2023-07-29T13:30:09.382 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:30:09.382 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=69}) v3 -- 0x7fdb900710b0 con 0x5612b54ab210 2023-07-29T13:30:09.383 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:30:11.387 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.377+0000 7fdba15f8640 20 client.4402.objecter 19 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:30:11.388 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.827+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:11.388 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.827+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:11.389 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:09.827+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:11.389 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 15 ==== osd_map(69..69 src has 1..69) v4 ==== 1404+0+0 (secure 0 0 0) 0x7fdb98087290 con 0x5612b54ab210 2023-07-29T13:30:11.389 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(69..69 src has 1..69) v4 2023-07-29T13:30:11.390 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [69,69] > 68 2023-07-29T13:30:11.390 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 69 2023-07-29T13:30:11.390 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 10 client.4402.objecter checking op 19 2023-07-29T13:30:11.390 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 69 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:11.391 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:11.391 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:30:11.391 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 10 monclient: _renew_subs 2023-07-29T13:30:11.391 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:30:11.391 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fdb900710b0 con 0x5612b54ab210 2023-07-29T13:30:11.391 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:30:11.392 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.425+0000 7fdba15f8640 20 client.4402.objecter 19 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:30:11.392 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.827+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:11.392 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.827+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:11.392 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:10.827+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:11.392 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.385+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 16 ==== osd_map(70..70 src has 1..70) v4 ==== 1428+0+0 (secure 0 0 0) 0x7fdb98087290 con 0x5612b54ab210 2023-07-29T13:30:11.392 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.385+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(70..70 src has 1..70) v4 2023-07-29T13:30:11.392 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.385+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [70,70] > 69 2023-07-29T13:30:11.393 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.385+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 70 2023-07-29T13:30:11.393 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.386+0000 7fdba15f8640 10 client.4402.objecter checking op 19 2023-07-29T13:30:11.393 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.386+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 70 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:11.393 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.386+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:11.393 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.386+0000 7fdba15f8640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:30:11.393 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.386+0000 7fdba15f8640 10 monclient: _renew_subs 2023-07-29T13:30:11.393 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.386+0000 7fdba15f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:30:11.860 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.386+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7fdb9008a100 con 0x5612b54ab210 2023-07-29T13:30:11.860 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.386+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:30:11.860 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.386+0000 7fdba15f8640 20 client.4402.objecter 19 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:30:11.860 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.827+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:11.861 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.827+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:11.861 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.827+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:11.861 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 27 ==== osd_op_reply(19 exist [writefull 0~5] v67'6 uv6 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:30:11.861 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(19 exist [writefull 0~5] v67'6 uv6 ondisk = 0) v8 2023-07-29T13:30:11.861 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:30:11.861 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 19 ondisk uv 6 in 7.1 attempt 0 2023-07-29T13:30:11.861 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:30:11.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 19 2023-07-29T13:30:11.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba45fe640 15 client.4402.objecter _finish_op 19 2023-07-29T13:30:11.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:30:11.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 19 2023-07-29T13:30:11.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:30:11.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba45fe640 20 librados: complete_aio_write 0x5612b55a5ee0 2023-07-29T13:30:11.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (3626 ms) 2023-07-29T13:30:11.862 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2023-07-29T13:30:11.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 20 librados: queue_aio_write 0x5612b538edf0 completion 0x5612b5562af0 write_seq 6 2023-07-29T13:30:11.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 10 client.4402.objecter _op_submit op 0x5612b5562600 2023-07-29T13:30:11.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 20 client.4402.objecter _calc_target epoch 70 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:11.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:11.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 10 client.4402.objecter _calc_target raw pgid 7.d3a59201 -> actual 7.1 acting [2,1] primary 2 2023-07-29T13:30:11.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 20 client.4402.objecter _get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:30:11.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 10 client.4402.objecter _op_submit oid exist '@7' '@7' [writefull 0~5 in=5b] tid 20 osd.2 2023-07-29T13:30:11.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 20 client.4402.objecter get_session s=0x5612b5330d90 osd=2 3 2023-07-29T13:30:11.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 15 client.4402.objecter _session_op_assign 2 20 2023-07-29T13:30:11.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 15 client.4402.objecter _send_op 20 to 7.1 on osd.2 2023-07-29T13:30:13.979 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- osd_op(unknown.0.0:20 7.1 7:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70) v8 -- 0x5612b5586180 con 0x5612b54a4ec0 2023-07-29T13:30:13.980 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:30:13.980 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:11.858+0000 7fdba6a34780 5 client.4402.objecter 1 in flight 2023-07-29T13:30:13.980 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 17 ==== osd_map(71..71 src has 1..71) v4 ==== 288+0+0 (secure 0 0 0) 0x7fdb98087290 con 0x5612b54ab210 2023-07-29T13:30:13.980 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(71..71 src has 1..71) v4 2023-07-29T13:30:13.980 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [71,71] > 70 2023-07-29T13:30:13.980 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 71 2023-07-29T13:30:13.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 10 client.4402.objecter checking op 20 2023-07-29T13:30:13.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 71 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:13.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:13.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:30:13.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 10 monclient: _renew_subs 2023-07-29T13:30:13.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:30:13.981 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7fdb9008a100 con 0x5612b54ab210 2023-07-29T13:30:13.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:30:13.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.427+0000 7fdba15f8640 20 client.4402.objecter 20 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:30:13.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.827+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:13.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.827+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:13.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:12.827+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:13.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.827+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:13.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.827+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:13.982 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.827+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:13.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.885+0000 7fdba6061640 10 client.4402.objecter tick 2023-07-29T13:30:13.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 <== mon.1 v2:172.21.15.45:3300/0 18 ==== osd_map(72..72 src has 1..72) v4 ==== 667+0+0 (secure 0 0 0) 0x7fdb980602d0 con 0x5612b54ab210 2023-07-29T13:30:13.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_map(72..72 src has 1..72) v4 2023-07-29T13:30:13.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map got epochs [72,72] > 71 2023-07-29T13:30:13.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 3 client.4402.objecter handle_osd_map decoding incremental epoch 72 2023-07-29T13:30:13.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 10 client.4402.objecter checking op 20 2023-07-29T13:30:13.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 20 client.4402.objecter _calc_target epoch 72 base exist @7 precalc_pgid 0 pgid 0.0 is_write 2023-07-29T13:30:13.983 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 20 client.4402.objecter _calc_target target exist @7 -> pgid 7.d3a59201 2023-07-29T13:30:14.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T13:30:14.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 10 monclient: _renew_subs 2023-07-29T13:30:14.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T13:30:14.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 1 -- 172.21.15.40:0/3644954068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7fdb9008a100 con 0x5612b54ab210 2023-07-29T13:30:14.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 20 client.4402.objecter dump_active .. 0 homeless 2023-07-29T13:30:14.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:13.977+0000 7fdba15f8640 20 client.4402.objecter 20 7.d3a59201 osd.2 exist [writefull 0~5 in=5b] 2023-07-29T13:30:14.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.827+0000 7fdb8bfff640 10 monclient: tick 2023-07-29T13:30:14.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.827+0000 7fdb8bfff640 10 monclient: _check_auth_tickets 2023-07-29T13:30:14.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.827+0000 7fdb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T13:30:14.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 <== osd.2 v2:172.21.15.40:6800/85572 28 ==== osd_op_reply(20 exist [writefull 0~5] v70'7 uv7 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fdb9c002830 con 0x5612b54a4ec0 2023-07-29T13:30:14.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 10 client.4402.objecter ms_dispatch 0x5612b5385550 osd_op_reply(20 exist [writefull 0~5] v70'7 uv7 ondisk = 0) v8 2023-07-29T13:30:14.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 10 client.4402.objecter in handle_osd_op_reply 2023-07-29T13:30:14.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 7 client.4402.objecter handle_osd_op_reply 20 ondisk uv 7 in 7.1 attempt 0 2023-07-29T13:30:14.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 10 client.4402.objecter op 0 rval 0 len 0 2023-07-29T13:30:14.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 15 client.4402.objecter handle_osd_op_reply completed tid 20 2023-07-29T13:30:14.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 15 client.4402.objecter _finish_op 20 2023-07-29T13:30:14.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 4 2023-07-29T13:30:14.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 15 client.4402.objecter _session_op_remove 2 20 2023-07-29T13:30:14.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 5 client.4402.objecter 0 in flight 2023-07-29T13:30:14.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 20 librados: complete_aio_write 0x5612b5562af0 2023-07-29T13:30:14.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (3006 ms) 2023-07-29T13:30:14.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 10 librados: watch_flush enter 2023-07-29T13:30:14.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 10 librados: watch_flush exit 2023-07-29T13:30:14.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 10 client.4402.objecter close_session for osd.2 2023-07-29T13:30:14.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x5612b54a4ec0 msgr2=0x5612b5499320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:30:14.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x5612b54a4ec0 0x5612b5499320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T13:30:14.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 20 client.4402.objecter put_session s=0x5612b5330d90 osd=2 1 2023-07-29T13:30:14.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 20 client.4402.objecter shutdown clearing up homeless session... 2023-07-29T13:30:14.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 10 client.4402.objecter successfully canceled tick 2023-07-29T13:30:14.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fdb9006a1d0 msgr2=0x7fdb9006c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:30:14.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fdb9006a1d0 0x7fdb9006c680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5612b5421bb0 tx=0x7fdb9404e070 comp rx=0 tx=0).stop 2023-07-29T13:30:14.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 10 monclient: shutdown 2023-07-29T13:30:14.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T13:30:14.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 msgr2=0x5612b54a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T13:30:14.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 1 --2- 172.21.15.40:0/3644954068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5612b54ab210 0x5612b54a4440 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800ade0 tx=0x7fdb98002f30 comp rx=0 tx=0).stop 2023-07-29T13:30:14.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba45fe640 1 -- 172.21.15.40:0/3644954068 reap_dead start 2023-07-29T13:30:14.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 shutdown_connections 2023-07-29T13:30:14.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 >> 172.21.15.40:0/3644954068 conn(0x5612b53db9b0 msgr2=0x5612b54a75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T13:30:14.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.864+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 shutdown_connections 2023-07-29T13:30:14.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.865+0000 7fdba6a34780 1 -- 172.21.15.40:0/3644954068 wait complete. 2023-07-29T13:30:14.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.865+0000 7fdba6a34780 1 librados: shutdown 2023-07-29T13:30:14.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [----------] 12 tests from AsioRados (31573 ms total) 2023-07-29T13:30:14.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: 2023-07-29T13:30:14.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [----------] Global test environment tear-down 2023-07-29T13:30:14.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (61120 ms total) 2023-07-29T13:30:14.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_asio: [ PASSED ] 12 tests. 2023-07-29T13:30:15.041 INFO:tasks.workunit.client.0.smithi040.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2023-07-29T13:30:15.041 INFO:tasks.workunit.client.0.smithi040.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2023-07-29T13:30:15.041 INFO:tasks.workunit.client.0.smithi040.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2023-07-29T13:30:15.042 INFO:tasks.workunit.client.0.smithi040.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2023-07-29T13:30:15.042 INFO:tasks.workunit.client.0.smithi040.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (61107 ms) 2023-07-29T13:30:15.042 INFO:tasks.workunit.client.0.smithi040.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (61107 ms total) 2023-07-29T13:30:15.042 INFO:tasks.workunit.client.0.smithi040.stdout: api_cls_remote_reads: 2023-07-29T13:30:15.042 INFO:tasks.workunit.client.0.smithi040.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2023-07-29T13:30:15.042 INFO:tasks.workunit.client.0.smithi040.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (61107 ms total) 2023-07-29T13:30:15.042 INFO:tasks.workunit.client.0.smithi040.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2023-07-29T13:30:27.059 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:03:22.393 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:27.102 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:03:22.436 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:27.178 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:03:22.512 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:27.196 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:03:22.530 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:27.238 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:03:22.572 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:27.239 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:03:22.524 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:27.254 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:03:22.538 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:27.332 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:03:22.666 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:27.363 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:03:22.647 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:46.788 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:03:42.122 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:46.792 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:03:42.126 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:46.845 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:03:42.129 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:46.847 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:03:42.131 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:46.871 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:03:42.205 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:46.875 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:03:42.209 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:46.896 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:03:42.231 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:46.942 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:03:42.226 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:46.969 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:03:42.304 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:30:59.012 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: Running main() from gmock_main.cc 2023-07-29T13:30:59.013 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2023-07-29T13:30:59.013 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [----------] Global test environment set-up. 2023-07-29T13:30:59.013 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2023-07-29T13:30:59.013 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2023-07-29T13:30:59.013 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (990 ms) 2023-07-29T13:30:59.013 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2023-07-29T13:30:59.013 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (86 ms) 2023-07-29T13:30:59.014 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2023-07-29T13:30:59.014 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (104099 ms) 2023-07-29T13:30:59.014 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (105175 ms total) 2023-07-29T13:30:59.014 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: 2023-07-29T13:30:59.014 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [----------] Global test environment tear-down 2023-07-29T13:30:59.014 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (105175 ms total) 2023-07-29T13:30:59.014 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2023-07-29T13:31:00.863 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2023-07-29T13:31:00.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2023-07-29T13:31:00.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2023-07-29T13:31:00.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2023-07-29T13:31:00.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2023-07-29T13:31:00.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: notify 2023-07-29T13:31:00.864 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (28300 ms) 2023-07-29T13:31:00.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2023-07-29T13:31:00.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (25818 ms) 2023-07-29T13:31:00.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (54118 ms total) 2023-07-29T13:31:00.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: 2023-07-29T13:31:00.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2023-07-29T13:31:00.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2023-07-29T13:31:00.865 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: notify 2023-07-29T13:31:00.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (23501 ms) 2023-07-29T13:31:00.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2023-07-29T13:31:00.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: notify 2023-07-29T13:31:00.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (5946 ms) 2023-07-29T13:31:00.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2023-07-29T13:31:00.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (1533 ms) 2023-07-29T13:31:00.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2023-07-29T13:31:00.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (209 ms) 2023-07-29T13:31:00.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2023-07-29T13:31:00.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: handle_notify cookie 94811475461024 notify_id 450971566082 notifier_gid 4708 2023-07-29T13:31:00.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (2862 ms) 2023-07-29T13:31:00.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2023-07-29T13:31:00.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: handle_notify cookie 94811476295664 notify_id 455266533378 notifier_gid 4708 2023-07-29T13:31:00.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (2345 ms) 2023-07-29T13:31:00.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2023-07-29T13:31:00.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: handle_notify cookie 94811476295664 notify_id 455266533377 notifier_gid 4708 2023-07-29T13:31:00.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (933 ms) 2023-07-29T13:31:00.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2023-07-29T13:31:00.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: handle_notify cookie 94811476295664 notify_id 455266533379 notifier_gid 4708 2023-07-29T13:31:00.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (480 ms) 2023-07-29T13:31:00.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2023-07-29T13:31:00.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: handle_notify cookie 94811476295664 notify_id 455266533378 notifier_gid 4708 2023-07-29T13:31:00.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (284 ms) 2023-07-29T13:31:00.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2023-07-29T13:31:00.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: handle_notify cookie 94811476295664 notify_id 455266533380 notifier_gid 4708 2023-07-29T13:31:00.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (255 ms) 2023-07-29T13:31:00.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2023-07-29T13:31:00.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: trying... 2023-07-29T13:31:00.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: handle_notify cookie 94811476295664 notify_id 455266533379 notifier_gid 4708 2023-07-29T13:31:00.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: timed out 2023-07-29T13:31:02.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: flu api_c_read_operations: Running main() from gmock_main.cc 2023-07-29T13:31:02.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2023-07-29T13:31:02.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [----------] Global test environment set-up. 2023-07-29T13:31:02.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2023-07-29T13:31:02.076 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2023-07-29T13:31:02.077 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2023-07-29T13:31:02.077 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2023-07-29T13:31:02.077 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (52412 ms) 2023-07-29T13:31:02.077 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2023-07-29T13:31:02.077 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (4252 ms) 2023-07-29T13:31:02.077 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2023-07-29T13:31:02.077 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (107 ms) 2023-07-29T13:31:02.077 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2023-07-29T13:31:02.078 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (24379 ms) 2023-07-29T13:31:02.078 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2023-07-29T13:31:02.078 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (2782 ms) 2023-07-29T13:31:02.078 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2023-07-29T13:31:02.078 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (953 ms) 2023-07-29T13:31:02.078 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2023-07-29T13:31:02.078 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (102 ms) 2023-07-29T13:31:02.079 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2023-07-29T13:31:02.079 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (892 ms) 2023-07-29T13:31:02.079 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2023-07-29T13:31:02.079 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (10049 ms) 2023-07-29T13:31:02.079 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2023-07-29T13:31:02.079 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (226 ms) 2023-07-29T13:31:02.080 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2023-07-29T13:31:02.080 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (245 ms) 2023-07-29T13:31:02.080 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2023-07-29T13:31:02.080 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (79 ms) 2023-07-29T13:31:02.080 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2023-07-29T13:31:02.080 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (390 ms) 2023-07-29T13:31:02.080 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2023-07-29T13:31:02.081 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (315 ms) 2023-07-29T13:31:02.081 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2023-07-29T13:31:02.081 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (193 ms) 2023-07-29T13:31:02.081 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2023-07-29T13:31:02.081 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (1067 ms) 2023-07-29T13:31:02.081 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (98443 ms total) 2023-07-29T13:31:02.081 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: 2023-07-29T13:31:02.081 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [----------] Global test environment tear-down 2023-07-29T13:31:02.082 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (108167 ms total) 2023-07-29T13:31:02.082 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2023-07-29T13:31:16.334 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2023-07-29T13:31:16.334 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [----------] Global test environment set-up. 2023-07-29T13:31:16.334 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [----------] 7 tests from LibRadosList 2023-07-29T13:31:16.335 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2023-07-29T13:31:16.335 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ OK ] LibRadosList.ListObjects (98206 ms) 2023-07-29T13:31:16.335 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2023-07-29T13:31:16.335 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (1188 ms) 2023-07-29T13:31:16.335 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2023-07-29T13:31:16.335 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: myset foo1,foo2,foo3 2023-07-29T13:31:16.335 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo1 2023-07-29T13:31:16.336 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo2 2023-07-29T13:31:16.336 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo3 2023-07-29T13:31:16.336 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: myset foo1,foo4,foo5 2023-07-29T13:31:16.336 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo4 2023-07-29T13:31:16.336 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo5 2023-07-29T13:31:16.336 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo1 2023-07-29T13:31:16.336 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: myset foo6,foo7 2023-07-29T13:31:16.337 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo7 2023-07-29T13:31:16.337 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo6 2023-07-29T13:31:16.337 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2023-07-29T13:31:16.337 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: ns1:foo4 2023-07-29T13:31:16.337 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: ns1:foo5 2023-07-29T13:31:16.338 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: ns2:foo7 2023-07-29T13:31:16.338 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: ns2:foo6 2023-07-29T13:31:16.338 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: ns1:foo1 2023-07-29T13:31:16.338 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: :foo1 2023-07-29T13:31:16.338 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: :foo2 2023-07-29T13:31:16.338 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: :foo3 2023-07-29T13:31:16.339 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (9804 ms) 2023-07-29T13:31:16.339 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2023-07-29T13:31:16.339 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 1 0 2023-07-29T13:31:16.339 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 10 0 2023-07-29T13:31:16.339 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 13 0 2023-07-29T13:31:16.339 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 7 0 2023-07-29T13:31:16.339 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 14 0 2023-07-29T13:31:16.340 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 0 0 2023-07-29T13:31:16.340 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 15 0 2023-07-29T13:31:16.340 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 11 0 2023-07-29T13:31:16.340 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 5 0 2023-07-29T13:31:16.340 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 8 0 2023-07-29T13:31:16.340 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 6 0 2023-07-29T13:31:16.340 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 3 0 2023-07-29T13:31:16.340 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 4 0 2023-07-29T13:31:16.341 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 12 0 2023-07-29T13:31:16.341 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 9 0 2023-07-29T13:31:16.341 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 2 0 2023-07-29T13:31:16.341 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2023-07-29T13:31:16.341 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (5251 ms) 2023-07-29T13:31:16.341 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2023-07-29T13:31:16.341 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: x cursor=MIN 2023-07-29T13:31:16.342 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=1 cursor=8:02547ec2:::1:head 2023-07-29T13:31:16.342 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=10 cursor=8:52ea6a34:::10:head 2023-07-29T13:31:16.342 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=13 cursor=8:566253c9:::13:head 2023-07-29T13:31:16.342 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=7 cursor=8:5c6b0b28:::7:head 2023-07-29T13:31:16.342 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=14 cursor=8:62a1935d:::14:head 2023-07-29T13:31:16.342 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=0 cursor=8:6cac518f:::0:head 2023-07-29T13:31:16.342 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=15 cursor=8:863748b0:::15:head 2023-07-29T13:31:16.342 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=11 cursor=8:89d3ae78:::11:head 2023-07-29T13:31:16.343 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=5 cursor=8:b29083e3:::5:head 2023-07-29T13:31:16.343 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=8 cursor=8:bd63b0f1:::8:head 2023-07-29T13:31:16.343 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=6 cursor=8:c4fdafeb:::6:head 2023-07-29T13:31:16.343 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=3 cursor=8:cfc208b3:::3:head 2023-07-29T13:31:16.343 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=4 cursor=8:d83876eb:::4:head 2023-07-29T13:31:16.343 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=12 cursor=8:de5d7c5f:::12:head 2023-07-29T13:31:16.343 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=9 cursor=8:e960b815:::9:head 2023-07-29T13:31:16.343 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > oid=2 cursor=8:f905c69b:::2:head 2023-07-29T13:31:16.344 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: FIRST> seek to MIN oid=1 2023-07-29T13:31:16.344 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=1 cursor=8:02547ec2:::1:head 2023-07-29T13:31:16.344 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:02547ec2:::1:head 2023-07-29T13:31:16.344 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:02547ec2:::1:head -> 1 2023-07-29T13:31:16.344 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=10 cursor=8:52ea6a34:::10:head 2023-07-29T13:31:16.344 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:52ea6a34:::10:head 2023-07-29T13:31:16.344 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:52ea6a34:::10:head -> 10 2023-07-29T13:31:16.345 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=13 cursor=8:566253c9:::13:head 2023-07-29T13:31:16.345 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:566253c9:::13:head 2023-07-29T13:31:16.345 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:566253c9:::13:head -> 13 2023-07-29T13:31:16.345 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=7 cursor=8:5c6b0b28:::7:head 2023-07-29T13:31:16.345 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:5c6b0b28:::7:head 2023-07-29T13:31:16.345 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:5c6b0b28:::7:head -> 7 2023-07-29T13:31:17.820 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=14 cursor=8:62a1935d:::14: api_cmd: Running main() from gmock_main.cc 2023-07-29T13:31:17.820 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2023-07-29T13:31:17.820 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [----------] Global test environment set-up. 2023-07-29T13:31:17.820 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2023-07-29T13:31:17.821 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2023-07-29T13:31:17.821 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (1127 ms) 2023-07-29T13:31:17.821 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2023-07-29T13:31:17.821 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (71 ms) 2023-07-29T13:31:17.821 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2023-07-29T13:31:17.821 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (104139 ms) 2023-07-29T13:31:17.821 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2023-07-29T13:31:17.822 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:30:55.015738+0000 mon.c [INF] from='client.? 172.21.15.40:0/1599494113' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi040-90886-6"}]: dispatch 2023-07-29T13:31:17.822 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:30:57.032934+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi040-90886-6"}]: dispatch 2023-07-29T13:31:17.822 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:00.629167+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi040-90886-6"}]': finished 2023-07-29T13:31:17.822 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:01.260233+0000 mon.c [INF] from='client.? 172.21.15.40:0/2193339159' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi040-90899-10"}]: dispatch 2023-07-29T13:31:17.822 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:01.492335+0000 mon.b [INF] from='client.? 172.21.15.40:0/2477781666' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2023-07-29T13:31:17.822 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:01.774323+0000 mon.c [INF] from='client.? 172.21.15.40:0/1599494113' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi040-90886-6", "tierpool": "test-rados-api-smithi040-90886-7"}]: dispatch 2023-07-29T13:31:17.822 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:01.825886+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi040-90899-10"}]: dispatch 2023-07-29T13:31:17.823 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:02.040693+0000 mon.a [INF] from='client.? 172.21.15.40:0/246842904' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi040-91142-12"}]: dispatch 2023-07-29T13:31:17.823 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:03.575271+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi040-90886-6", "tierpool": "test-rados-api-smithi040-90886-7"}]: dispatch 2023-07-29T13:31:17.823 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:03.812059+0000 client.admin [INF] onexx 2023-07-29T13:31:17.823 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:03.835721+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2023-07-29T13:31:17.823 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:04.627533+0000 mon.a [INF] from='client.? 172.21.15.40:0/246842904' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi040-91142-12"}]: dispatch 2023-07-29T13:31:17.823 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:08.367204+0000 mon.b [INF] from='client.? 172.21.15.40:0/2477781666' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2023-07-29T13:31:17.823 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:08.429319+0000 mon.b [INF] from='client.? 172.21.15.40:0/1604209023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi040-90756-22"}]: dispatch 2023-07-29T13:31:17.823 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:08.440824+0000 mon.c [INF] from='client.? 172.21.15.40:0/3975748423' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi040-90744-16"}]: dispatch 2023-07-29T13:31:18.770 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:08.695529+0000 mon.c [INF] from='client.? 172.21.15.40:0/2193339159' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi040-90899-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-07-29T13:31:18.771 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:08.989948+0000 mon.b [INF] from='client.? 172.21.15.40:0/1542284388' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi040-91064-7"}]: dispatch 2023-07-29T13:31:18.771 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:09.188198+0000 mon.c [INF] from='client.? 172.21.15.40:0/271687670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi040-91417-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-07-29T13:31:18.771 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:09.749883+0000 mon.a [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-07-29T13:31:18.771 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:10.107018+0000 client.admin [INF] twoxx 2023-07-29T13:31:18.771 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:10.132445+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2023-07-29T13:31:18.771 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:10.136116+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi040-90756-22"}]: dispatch 2023-07-29T13:31:18.772 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:10.146655+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi040-91064-7"}]: dispatch 2023-07-29T13:31:18.772 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:10.242229+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi040-90744-16"}]: dispatch 2023-07-29T13:31:18.772 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:10.434177+0000 mon.a [INF] from='client.? 172.21.15.40:0/246842904' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi040-91142-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2023-07-29T13:31:18.772 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:10.444185+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi040-91123-7"}]': finished 2023-07-29T13:31:18.772 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:10.447158+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi040-90832-10"}]': finished 2023-07-29T13:31:18.773 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:10.582467+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi040-90899-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-07-29T13:31:18.773 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:10.616515+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi040-91417-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-07-29T13:31:18.773 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:11.559443+0000 mon.a [INF] from='client.? 172.21.15.40:0/246842904' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi040-91142-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi040-91142-12"}]: dispatch 2023-07-29T13:31:18.774 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:11.636661+0000 mon.c [INF] from='client.? 172.21.15.40:0/3975748423' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi040-90744-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-07-29T13:31:18.774 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:11.837302+0000 mon.c [INF] from='client.? 172.21.15.40:0/1965180979' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi040-91123-7"}]: dispatch 2023-07-29T13:31:20.466 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: Running main() from gmock_main.cc 2023-07-29T13:31:20.466 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2023-07-29T13:31:20.467 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [----------] Global test environment set-up. 2023-07-29T13:31:20.467 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [----------] 8 tests from LibRadosLock 2023-07-29T13:31:20.467 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2023-07-29T13:31:20.467 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (13398 ms) 2023-07-29T13:31:20.467 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2023-07-29T13:31:20.467 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLock.LockShared (7250 ms) 2023-07-29T13:31:20.467 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2023-07-29T13:31:20.468 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1650 ms) 2023-07-29T13:31:20.468 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2023-07-29T13:31:20.468 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (4406 ms) 2023-07-29T13:31:20.468 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2023-07-29T13:31:20.468 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (4537 ms) 2023-07-29T13:31:20.468 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2023-07-29T13:31:20.468 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLock.Unlock (28556 ms) 2023-07-29T13:31:20.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2023-07-29T13:31:20.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (2773 ms) 2023-07-29T13:31:20.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2023-07-29T13:31:20.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (2654 ms) 2023-07-29T13:31:20.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [----------] 8 tests from LibRadosLock (65224 ms total) 2023-07-29T13:31:20.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: 2023-07-29T13:31:20.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2023-07-29T13:31:20.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2023-07-29T13:31:20.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (21811 ms) 2023-07-29T13:31:20.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2023-07-29T13:31:20.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (825 ms) 2023-07-29T13:31:20.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2023-07-29T13:31:20.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1803 ms) 2023-07-29T13:31:20.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2023-07-29T13:31:20.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (2557 ms) 2023-07-29T13:31:20.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2023-07-29T13:31:20.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (806 ms) 2023-07-29T13:31:20.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2023-07-29T13:31:20.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (261 ms) 2023-07-29T13:31:20.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2023-07-29T13:31:20.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (497 ms) 2023-07-29T13:31:20.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2023-07-29T13:31:20.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (236 ms) 2023-07-29T13:31:20.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (28796 ms total) 2023-07-29T13:31:20.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: 2023-07-29T13:31:20.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [----------] Global test environment tear-down 2023-07-29T13:31:20.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (126792 ms total) 2023-07-29T13:31:20.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock: [ PASSED ] 16 tests. 2023-07-29T13:31:20.557 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: Running main() from gmock_main.cc 2023-07-29T13:31:20.557 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2023-07-29T13:31:20.557 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [----------] Global test environment set-up. 2023-07-29T13:31:20.557 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2023-07-29T13:31:20.558 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: seed 91123 2023-07-29T13:31:20.558 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2023-07-29T13:31:20.558 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (28341 ms) 2023-07-29T13:31:20.558 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2023-07-29T13:31:20.558 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (3340 ms) 2023-07-29T13:31:20.558 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2023-07-29T13:31:20.558 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (123 ms) 2023-07-29T13:31:20.559 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2023-07-29T13:31:20.559 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (5363 ms) 2023-07-29T13:31:20.559 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2023-07-29T13:31:20.559 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (23380 ms) 2023-07-29T13:31:20.559 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (60547 ms total) 2023-07-29T13:31:20.559 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: 2023-07-29T13:31:20.559 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2023-07-29T13:31:20.559 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2023-07-29T13:31:20.560 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (27723 ms) 2023-07-29T13:31:20.560 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2023-07-29T13:31:20.560 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1832 ms) 2023-07-29T13:31:20.560 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2023-07-29T13:31:20.560 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (2604 ms) 2023-07-29T13:31:20.560 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2023-07-29T13:31:20.560 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (2260 ms) 2023-07-29T13:31:20.560 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (34419 ms total) 2023-07-29T13:31:20.561 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: 2023-07-29T13:31:20.561 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [----------] Global test environment tear-down 2023-07-29T13:31:20.561 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (126756 ms total) 2023-07-29T13:31:20.561 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat_pp: [ PASSED ] 9 tests. 2023-07-29T13:31:20.660 INFO:tasks.workunit.client.0.smithi040.stdout:head 2023-07-29T13:31:20.660 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:62a1935d:::14:head 2023-07-29T13:31:20.661 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:62a1935d:::14:head -> 14 2023-07-29T13:31:20.661 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=0 cursor=8:6cac518f:::0:head 2023-07-29T13:31:20.661 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:6cac518f:::0:head 2023-07-29T13:31:20.661 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:6cac518f:::0:head -> 0 2023-07-29T13:31:20.661 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=15 cursor=8:863748b0:::15:head 2023-07-29T13:31:20.661 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:863748b0:::15:head 2023-07-29T13:31:20.661 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:863748b0:::15:head -> 15 2023-07-29T13:31:20.661 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=11 cursor=8:89d3ae78:::11:head 2023-07-29T13:31:20.662 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:89d3ae78:::11:head 2023-07-29T13:31:20.662 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:89d3ae78:::11:head -> 11 2023-07-29T13:31:20.662 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=5 cursor=8:b29083e3:::5:head 2023-07-29T13:31:20.662 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:b29083e3:::5:head 2023-07-29T13:31:20.662 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:b29083e3:::5:head -> 5 2023-07-29T13:31:20.662 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=8 cursor=8:bd63b0f1:::8:head 2023-07-29T13:31:20.662 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:bd63b0f1:::8:head 2023-07-29T13:31:20.663 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:bd63b0f1:::8:head -> 8 2023-07-29T13:31:20.663 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=6 cursor=8:c4fdafeb:::6:head 2023-07-29T13:31:20.663 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:c4fdafeb:::6:head 2023-07-29T13:31:20.663 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:c4fdafeb:::6:head -> 6 2023-07-29T13:31:20.663 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=3 cursor=8:cfc208b3:::3:head 2023-07-29T13:31:20.663 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:cfc208b3:::3:head 2023-07-29T13:31:20.663 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:cfc208b3:::3:head -> 3 2023-07-29T13:31:20.663 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=4 cursor=8:d83876eb:::4:head 2023-07-29T13:31:20.664 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:d83876eb:::4:head 2023-07-29T13:31:20.664 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:d83876eb:::4:head -> 4 2023-07-29T13:31:20.664 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=12 cursor=8:de5d7c5f:::12:head 2023-07-29T13:31:20.664 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:de5d7c5f:::12:head 2023-07-29T13:31:20.664 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:de5d7c5f:::12:head -> 12 2023-07-29T13:31:20.664 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=9 cursor=8:e960b815:::9:head 2023-07-29T13:31:20.664 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:e960b815:::9:head 2023-07-29T13:31:20.664 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:e960b815:::9:head -> 9 2023-07-29T13:31:20.665 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : oid=2 cursor=8:f905c69b:::2:head 2023-07-29T13:31:20.665 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:f905c69b:::2:head 2023-07-29T13:31:20.665 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:f905c69b:::2:head -> 2 2023-07-29T13:31:20.665 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:5c6b0b28:::7:head 2023-07-29T13:31:20.665 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:5c6b0b28:::7:head expected=8:5c6b0b28:::7:head 2023-07-29T13:31:20.665 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:5c6b0b28:::7:head -> 7 2023-07-29T13:31:20.665 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=7 expected=7 2023-07-29T13:31:20.666 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:52ea6a34:::10:head 2023-07-29T13:31:20.666 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:52ea6a34:::10:head expected=8:52ea6a34:::10:head 2023-07-29T13:31:20.666 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:52ea6a34:::10:head -> 10 2023-07-29T13:31:20.666 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=10 expected=10 2023-07-29T13:31:20.666 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:863748b0:::15:head 2023-07-29T13:31:20.666 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:863748b0:::15:head expected=8:863748b0:::15:head 2023-07-29T13:31:20.666 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:863748b0:::15:head -> 15 2023-07-29T13:31:20.666 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=15 expected=15 2023-07-29T13:31:20.667 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:6cac518f:::0:head 2023-07-29T13:31:20.667 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:6cac518f:::0:head expected=8:6cac518f:::0:head 2023-07-29T13:31:20.667 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:6cac518f:::0:head -> 0 2023-07-29T13:31:20.667 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=0 expected=0 2023-07-29T13:31:20.667 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:62a1935d:::14:head 2023-07-29T13:31:20.667 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:62a1935d:::14:head expected=8:62a1935d:::14:head 2023-07-29T13:31:20.667 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:62a1935d:::14:head -> 14 2023-07-29T13:31:20.667 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=14 expected=14 2023-07-29T13:31:20.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:566253c9:::13:head 2023-07-29T13:31:20.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:566253c9:::13:head expected=8:566253c9:::13:head 2023-07-29T13:31:20.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:566253c9:::13:head -> 13 2023-07-29T13:31:20.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=13 expected=13 2023-07-29T13:31:20.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:f905c69b:::2:head 2023-07-29T13:31:20.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:f905c69b:::2:head expected=8:f905c69b:::2:head 2023-07-29T13:31:20.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:f905c69b:::2:head -> 2 2023-07-29T13:31:20.668 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=2 expected=2 2023-07-29T13:31:20.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:e960b815:::9:head 2023-07-29T13:31:20.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:e960b815:::9:head expected=8:e960b815:::9:head 2023-07-29T13:31:20.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:e960b815:::9:head -> 9 2023-07-29T13:31:20.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=9 expected=9 2023-07-29T13:31:20.669 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:de5d7c5f:::12:head 2023-07-29T13:31:22.101 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:11.996959+0000 mon.b [INF] from='client.? 172.21.15.40:0/4276256865' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi040-90832-10"}]: dispatch 2023-07-29T13:31:22.101 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:12.006176+0000 mon.b [INF] from='client.? 172.21.15.40:0/1604209023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi040-90756-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-07-29T13:31:22.101 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:12.302340+0000 mon.b [INF] from='client.? 172.21.15.40:0/2702628287' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi040-90704-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-07-29T13:31:22.102 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:12.425691+0000 mon.b [INF] from='client.? 172.21.15.40:0/2809916181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi040-90728-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-07-29T13:31:22.102 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:13.691770+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi040-90744-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-07-29T13:31:22.102 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:13.707691+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi040-91123-7"}]: dispatch 2023-07-29T13:31:22.102 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:13.836277+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi040-90832-10"}]: dispatch 2023-07-29T13:31:22.102 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:13.852753+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi040-90756-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-07-29T13:31:22.102 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:13.882921+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi040-90704-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-07-29T13:31:22.102 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:13.898579+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi040-90728-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-07-29T13:31:22.103 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:14.039891+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi040-91064-7"}]': finished 2023-07-29T13:31:22.103 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:14.043989+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi040-90899-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2023-07-29T13:31:22.103 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:14.047011+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi040-91417-5","app": "rados","yes_i_really_mean_it": true}]': finished 2023-07-29T13:31:22.103 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:14.111551+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2023-07-29T13:31:22.103 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:14.482964+0000 mon.b [INF] from='client.? 172.21.15.40:0/776358317' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi040-91211-4"}]: dispatch 2023-07-29T13:31:22.103 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:15.195631+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi040-91211-4"}]: dispatch 2023-07-29T13:31:23.373 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: g api_stat: Running main() from gmock_main.cc 2023-07-29T13:31:23.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2023-07-29T13:31:23.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [----------] Global test environment set-up. 2023-07-29T13:31:23.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [----------] 5 tests from LibRadosStat 2023-07-29T13:31:23.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2023-07-29T13:31:23.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ OK ] LibRadosStat.Stat (28533 ms) 2023-07-29T13:31:23.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2023-07-29T13:31:23.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (3920 ms) 2023-07-29T13:31:23.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2023-07-29T13:31:23.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ OK ] LibRadosStat.StatNS (32266 ms) 2023-07-29T13:31:23.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2023-07-29T13:31:23.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (15 ms) 2023-07-29T13:31:23.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2023-07-29T13:31:23.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2633 ms) 2023-07-29T13:31:23.376 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [----------] 5 tests from LibRadosStat (67367 ms total) 2023-07-29T13:31:23.376 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: 2023-07-29T13:31:23.376 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2023-07-29T13:31:23.376 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2023-07-29T13:31:23.376 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (21723 ms) 2023-07-29T13:31:23.376 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2023-07-29T13:31:23.377 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (4005 ms) 2023-07-29T13:31:23.377 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2023-07-29T13:31:23.377 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (2164 ms) 2023-07-29T13:31:23.377 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2023-07-29T13:31:23.377 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (3254 ms) 2023-07-29T13:31:23.377 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (31146 ms total) 2023-07-29T13:31:23.378 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: 2023-07-29T13:31:23.378 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [----------] Global test environment tear-down 2023-07-29T13:31:23.378 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (129596 ms total) 2023-07-29T13:31:23.378 INFO:tasks.workunit.client.0.smithi040.stdout: api_stat: [ PASSED ] 9 tests. 2023-07-29T13:31:24.524 INFO:tasks.workunit.client.0.smithi040.stdout:ot: 2023-07-29T13:31:15.531600+0000 mon.b [INF] from='client.? 172.21.15.40:0/1542284388' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi040-91064-7"}]: dispatch 2023-07-29T13:31:24.525 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:15.719401+0000 mon.c [INF] from='client.? 172.21.15.40:0/2193339159' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi040-90899-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi040-90899-10"}]: dispatch 2023-07-29T13:31:24.525 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:16.914335+0000 mon.a [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-07-29T13:31:24.525 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:17.005549+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi040-90899-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi040-90899-10"}]: dispatch 2023-07-29T13:31:24.525 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:17.088773+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi040-91064-7"}]: dispatch 2023-07-29T13:31:24.525 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:17.469131+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi040-90744-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2023-07-29T13:31:24.526 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:17.471749+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi040-91123-7"}]': finished 2023-07-29T13:31:24.526 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:17.475221+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi040-90832-10"}]': finished 2023-07-29T13:31:24.526 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:17.477011+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi040-90756-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2023-07-29T13:31:24.526 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:17.479926+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi040-90704-6","app": "rados","yes_i_really_mean_it": true}]': finished 2023-07-29T13:31:24.526 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:17.482889+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi040-90728-6","app": "rados","yes_i_really_mean_it": true}]': finished 2023-07-29T13:31:24.526 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:17.529679+0000 mon.b [INF] from='client.? 172.21.15.40:0/2477781666' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2023-07-29T13:31:24.526 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:18.591670+0000 mon.c [INF] from='client.? 172.21.15.40:0/3975748423' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi040-90744-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi040-90744-16"}]: dispatch 2023-07-29T13:31:24.526 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:18.841656+0000 client.admin [INF] threexx 2023-07-29T13:31:24.527 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:19.145919+0000 mon.b [INF] from='client.? 172.21.15.40:0/1604209023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi040-90756-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi040-90756-22"}]: dispatch 2023-07-29T13:31:24.527 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:19.172058+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2023-07-29T13:31:32.387 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:20.608781+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRados list_parallel: process_1_[91445]: starting. 2023-07-29T13:31:32.387 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_1_[91445]: creating pool ceph_test_rados_list_parallel.smithi040-91409 2023-07-29T13:31:32.387 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_1_[91445]: created object 0... 2023-07-29T13:31:32.387 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_1_[91445]: created object 25... 2023-07-29T13:31:32.387 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_1_[91445]: created object 49... 2023-07-29T13:31:32.388 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_1_[91445]: finishing. 2023-07-29T13:31:32.388 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_1_[91445]: shutting down. 2023-07-29T13:31:32.388 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_2_[91446]: starting. 2023-07-29T13:31:32.388 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_2_[91446]: listing objects. 2023-07-29T13:31:32.388 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_2_[91446]: listed object 0... 2023-07-29T13:31:32.388 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_2_[91446]: listed object 25... 2023-07-29T13:31:32.388 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_2_[91446]: saw 50 objects 2023-07-29T13:31:32.389 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_2_[91446]: shutting down. 2023-07-29T13:31:32.389 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:31:32.389 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_3_[108170]: starting. 2023-07-29T13:31:32.389 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_3_[108170]: creating pool ceph_test_rados_list_parallel.smithi040-91409 2023-07-29T13:31:32.389 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_3_[108170]: created object 0... 2023-07-29T13:31:32.389 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_3_[108170]: created object 25... 2023-07-29T13:31:32.389 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_3_[108170]: created object 49... 2023-07-29T13:31:32.389 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_3_[108170]: finishing. 2023-07-29T13:31:32.390 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_3_[108170]: shutting down. 2023-07-29T13:31:32.390 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:31:32.390 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_4_[108171]: starting. 2023-07-29T13:31:32.390 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_4_[108171]: listing objects. 2023-07-29T13:31:32.390 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_4_[108171]: listed object 0... 2023-07-29T13:31:32.390 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_4_[108171]: listed object 25... 2023-07-29T13:31:32.390 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_4_[108171]: saw 46 objects 2023-07-29T13:31:32.390 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_4_[108171]: shutting down. 2023-07-29T13:31:32.391 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:31:32.391 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_5_[108172]: starting. 2023-07-29T13:31:32.391 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_5_[108172]: removed 25 objects... 2023-07-29T13:31:32.391 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_5_[108172]: removed half of the objects 2023-07-29T13:31:32.391 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_5_[108172]: removed 50 objects... 2023-07-29T13:31:32.391 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_5_[108172]: removed 50 objects 2023-07-29T13:31:32.391 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_5_[108172]: shutting down. 2023-07-29T13:31:32.391 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:31:32.392 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:31:32.392 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_6_[116793]: starting. 2023-07-29T13:31:32.392 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_6_[116793]: creating pool ceph_test_rados_list_parallel.smithi040-91409 2023-07-29T13:31:32.392 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_6_[116793]: created object 0... 2023-07-29T13:31:32.392 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_6_[116793]: created object 25... 2023-07-29T13:31:32.392 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_6_[116793]: created object 49... 2023-07-29T13:31:32.393 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_6_[116793]: finishing. 2023-07-29T13:31:32.393 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_6_[116793]: shutting down. 2023-07-29T13:31:32.393 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:31:32.393 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:31:32.393 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_7_[116794]: starting. 2023-07-29T13:31:32.393 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_7_[116794]: listing objects. 2023-07-29T13:31:32.393 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_7_[116794]: listed object 0... 2023-07-29T13:31:32.394 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_7_[116794]: listed object 25... 2023-07-29T13:31:32.394 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_7_[116794]: listed object 50... 2023-07-29T13:31:32.394 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_7_[116794]: saw 55 objects 2023-07-29T13:31:32.394 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_7_[116794]: shutting down. 2023-07-29T13:31:32.394 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:31:32.394 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:31:32.394 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_8_[116795]: starting. 2023-07-29T13:31:32.394 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_8_[116795]: added 25 objects... 2023-07-29T13:31:32.395 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_8_[116795]: added half of the objects 2023-07-29T13:31:32.395 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_8_[116795]: added 50 objects... 2023-07-29T13:31:32.395 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_8_[116795]: added 50 objects 2023-07-29T13:31:32.395 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_8_[116795]: shutting down. 2023-07-29T13:31:32.395 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:31:32.395 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:31:34.504 INFO:tasks.workunit.client.0.smithi040.stdout: IoECPP_smithi040-90756-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi040-90756-22"}]: dispatch 2023-07-29T13:31:34.505 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:20.723809+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi040-90744-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi040-90744-16"}]: dispatch 2023-07-29T13:31:34.505 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:20.947725+0000 mon.a [INF] from='client.? 172.21.15.40:0/246842904' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi040-91142-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi040-91142-12"}]': finished 2023-07-29T13:31:34.505 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:20.949557+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi040-91211-4"}]': finished 2023-07-29T13:31:34.505 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: got: 2023-07-29T13:31:20.961909+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi040-91064-7"}]': finished 2023-07-29T13:31:34.505 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (35288 ms) 2023-07-29T13:31:34.505 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (140625 ms total) 2023-07-29T13:31:34.505 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: 2023-07-29T13:31:34.506 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [----------] Global test environment tear-down 2023-07-29T13:31:34.506 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (140625 ms total) 2023-07-29T13:31:34.506 INFO:tasks.workunit.client.0.smithi040.stdout: api_cmd: [ PASSED ] 4 tests. 2023-07-29T13:31:40.398 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: Running main() from gmock_main.cc 2023-07-29T13:31:40.398 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2023-07-29T13:31:40.398 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [----------] Global test environment set-up. 2023-07-29T13:31:40.398 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2023-07-29T13:31:40.398 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: seed 90756 2023-07-29T13:31:40.399 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2023-07-29T13:31:40.399 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2023-07-29T13:31:40.399 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2023-07-29T13:31:40.399 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (25495 ms) 2023-07-29T13:31:40.399 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2023-07-29T13:31:40.399 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (2181 ms) 2023-07-29T13:31:40.399 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2023-07-29T13:31:40.399 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (1716 ms) 2023-07-29T13:31:40.400 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2023-07-29T13:31:40.400 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (199 ms) 2023-07-29T13:31:40.400 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2023-07-29T13:31:40.400 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (2614 ms) 2023-07-29T13:31:40.400 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2023-07-29T13:31:40.400 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (3809 ms) 2023-07-29T13:31:40.400 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2023-07-29T13:31:40.401 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (12796 ms) 2023-07-29T13:31:40.401 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2023-07-29T13:31:40.401 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (4436 ms) 2023-07-29T13:31:40.401 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2023-07-29T13:31:40.401 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (4364 ms) 2023-07-29T13:31:40.401 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2023-07-29T13:31:40.401 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (9524 ms) 2023-07-29T13:31:40.401 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2023-07-29T13:31:40.402 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (17113 ms) 2023-07-29T13:31:40.402 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2023-07-29T13:31:40.402 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (1193 ms) 2023-07-29T13:31:40.402 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2023-07-29T13:31:40.402 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (717 ms) 2023-07-29T13:31:40.402 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2023-07-29T13:31:40.402 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (1159 ms) 2023-07-29T13:31:40.403 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2023-07-29T13:31:40.403 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (9761 ms) 2023-07-29T13:31:40.403 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2023-07-29T13:31:40.403 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (131 ms) 2023-07-29T13:31:40.403 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2023-07-29T13:31:40.403 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (127 ms) 2023-07-29T13:31:40.403 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2023-07-29T13:31:40.403 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (107 ms) 2023-07-29T13:31:40.404 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2023-07-29T13:31:40.404 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (187 ms) 2023-07-29T13:31:40.404 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (97630 ms total) 2023-07-29T13:31:40.404 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: 2023-07-29T13:31:40.404 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2023-07-29T13:31:40.404 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2023-07-29T13:31:40.404 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (6896 ms) 2023-07-29T13:31:40.405 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2023-07-29T13:31:40.405 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (1152 ms) 2023-07-29T13:31:40.405 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2023-07-29T13:31:40.405 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (134 ms) 2023-07-29T13:31:40.405 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2023-07-29T13:31:40.405 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (1277 ms) 2023-07-29T13:31:40.405 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2023-07-29T13:31:43.345 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.shing 2023-07-29T13:31:43.345 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: flushed 2023-07-29T13:31:43.345 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3166 ms) 2023-07-29T13:31:43.345 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2023-07-29T13:31:43.345 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: trying... 2023-07-29T13:31:43.346 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: handle_notify cookie 94811476295664 notify_id 463856467970 notifier_gid 4708 2023-07-29T13:31:43.346 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: timed out 2023-07-29T13:31:43.346 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: flushing 2023-07-29T13:31:43.346 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: flushed 2023-07-29T13:31:43.346 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (6291 ms) 2023-07-29T13:31:43.346 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2023-07-29T13:31:43.346 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: List watches 2023-07-29T13:31:43.347 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: notify2 2023-07-29T13:31:43.347 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: handle_notify cookie 94811476295664 notify_id 468151435268 notifier_gid 4708 2023-07-29T13:31:43.347 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: notify2 done 2023-07-29T13:31:43.347 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: watch_check 2023-07-29T13:31:43.347 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: unwatch2 2023-07-29T13:31:43.347 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: flushing 2023-07-29T13:31:43.347 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: done 2023-07-29T13:31:43.348 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3273 ms) 2023-07-29T13:31:43.348 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2023-07-29T13:31:43.348 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: List watches 2023-07-29T13:31:43.348 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: notify2 2023-07-29T13:31:43.348 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: handle_notify cookie 94811476295664 notify_id 472446402563 notifier_gid 4708 2023-07-29T13:31:43.348 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: notify2 done 2023-07-29T13:31:43.348 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: watch_check 2023-07-29T13:31:43.348 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: unwatch2 2023-07-29T13:31:43.349 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: flushing 2023-07-29T13:31:43.349 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: done 2023-07-29T13:31:43.349 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3285 ms) 2023-07-29T13:31:43.349 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (54363 ms total) 2023-07-29T13:31:43.349 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: 2023-07-29T13:31:43.349 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2023-07-29T13:31:43.349 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (149507 ms total) 2023-07-29T13:31:43.349 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2023-07-29T13:31:43.718 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_1_[91449]: starting. 2023-07-29T13:31:43.718 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_1_[91449]: creating pool ceph_test_rados_open_pools_parallel.smithi040-91437 2023-07-29T13:31:43.718 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_1_[91449]: created object 0... 2023-07-29T13:31:43.718 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_1_[91449]: created object 25... 2023-07-29T13:31:43.719 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_1_[91449]: created object 49... 2023-07-29T13:31:43.719 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_1_[91449]: finishing. 2023-07-29T13:31:43.719 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_1_[91449]: shutting down. 2023-07-29T13:31:43.719 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_2_[91450]: starting. 2023-07-29T13:31:43.719 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_2_[91450]: rados_pool_create. 2023-07-29T13:31:43.719 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_2_[91450]: rados_ioctx_create. 2023-07-29T13:31:43.719 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_2_[91450]: shutting down. 2023-07-29T13:31:43.720 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: ******************************* 2023-07-29T13:31:43.720 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_3_[110868]: starting. 2023-07-29T13:31:43.720 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_3_[110868]: creating pool ceph_test_rados_open_pools_parallel.smithi040-91437 2023-07-29T13:31:43.720 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_3_[110868]: created object 0... 2023-07-29T13:31:43.720 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_3_[110868]: created object 25... 2023-07-29T13:31:43.720 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_3_[110868]: created object 49... 2023-07-29T13:31:43.720 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_3_[110868]: finishing. 2023-07-29T13:31:43.720 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_3_[110868]: shutting down. 2023-07-29T13:31:43.721 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: ******************************* 2023-07-29T13:31:43.721 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_4_[110869]: starting. 2023-07-29T13:31:43.721 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_4_[110869]: rados_pool_create. 2023-07-29T13:31:43.721 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_4_[110869]: rados_ioctx_create. 2023-07-29T13:31:43.721 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: process_4_[110869]: shutting down. 2023-07-29T13:31:43.721 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: ******************************* 2023-07-29T13:31:43.721 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: ******************************* 2023-07-29T13:31:43.721 INFO:tasks.workunit.client.0.smithi040.stdout: open_pools_parallel: ******* SUCCESS ********** 2023-07-29T13:31:46.213 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_1_[91543]: starting. 2023-07-29T13:31:46.213 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_1_[91543]: creating pool ceph_test_rados_delete_pools_parallel.smithi040-91502 2023-07-29T13:31:46.213 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_1_[91543]: created object 0... 2023-07-29T13:31:46.214 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_1_[91543]: created object 25... 2023-07-29T13:31:46.214 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_1_[91543]: created object 49... 2023-07-29T13:31:46.214 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_1_[91543]: finishing. 2023-07-29T13:31:46.214 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_1_[91543]: shutting down. 2023-07-29T13:31:46.214 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_2_[91544]: starting. 2023-07-29T13:31:46.214 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_2_[91544]: deleting pool ceph_test_rados_delete_pools_parallel.smithi040-91502 2023-07-29T13:31:46.214 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_2_[91544]: shutting down. 2023-07-29T13:31:46.214 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: ******************************* 2023-07-29T13:31:46.215 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_3_[111444]: starting. 2023-07-29T13:31:46.215 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_3_[111444]: creating pool ceph_test_rados_delete_pools_parallel.smithi040-91502 2023-07-29T13:31:46.215 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_3_[111444]: created object 0... 2023-07-29T13:31:46.215 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_3_[111444]: created object 25... 2023-07-29T13:31:46.215 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_3_[111444]: created object 49... 2023-07-29T13:31:46.215 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_3_[111444]: finishing. 2023-07-29T13:31:46.215 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_3_[111444]: shutting down. 2023-07-29T13:31:46.216 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: ******************************* 2023-07-29T13:31:46.216 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_5_[111446]: starting. 2023-07-29T13:31:46.216 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_5_[111446]: listing objects. 2023-07-29T13:31:46.216 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_5_[111446]: listed object 0... 2023-07-29T13:31:46.216 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_5_[111446]: listed object 25... 2023-07-29T13:31:46.216 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_5_[111446]: saw 50 objects 2023-07-29T13:31:46.217 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_5_[111446]: shutting down. 2023-07-29T13:31:46.217 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: ******************************* 2023-07-29T13:31:46.217 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_4_[111445]: starting. 2023-07-29T13:31:46.217 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_4_[111445]: deleting pool ceph_test_rados_delete_pools_parallel.smithi040-91502 2023-07-29T13:31:46.217 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: process_4_[111445]: shutting down. 2023-07-29T13:31:46.217 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: ******************************* 2023-07-29T13:31:46.217 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: ******************************* 2023-07-29T13:31:46.218 INFO:tasks.workunit.client.0.smithi040.stdout: delete_pools_parallel: ******* SUCCESS ********** 2023-07-29T13:31:50.947 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: Running main() from gmock_main.cc 2023-07-29T13:31:50.947 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2023-07-29T13:31:50.947 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [----------] Global test environment set-up. 2023-07-29T13:31:50.948 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2023-07-29T13:31:50.948 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2023-07-29T13:31:50.948 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify_test_cb 2023-07-29T13:31:50.949 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (27556 ms) 2023-07-29T13:31:50.949 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2023-07-29T13:31:50.949 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2023-07-29T13:31:50.949 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94684139575584 err -107 2023-07-29T13:31:50.950 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (22903 ms) 2023-07-29T13:31:50.950 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2023-07-29T13:31:50.950 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2023-07-29T13:31:50.951 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94684141101328 err -107 2023-07-29T13:31:50.951 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (7056 ms) 2023-07-29T13:31:50.951 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2023-07-29T13:31:50.951 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_cb from 4373 notify_id 313532612608 cookie 94684141101328 2023-07-29T13:31:50.952 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (186 ms) 2023-07-29T13:31:50.952 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2023-07-29T13:31:50.952 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_cb from 4373 notify_id 317827579904 cookie 94684141101328 2023-07-29T13:31:50.953 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (2027 ms) 2023-07-29T13:31:50.953 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2023-07-29T13:31:50.953 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_cb from 4373 notify_id 322122547201 cookie 94684141093312 2023-07-29T13:31:50.953 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (1314 ms) 2023-07-29T13:31:50.954 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2023-07-29T13:31:50.954 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_cb from 4373 notify_id 386547056640 cookie 94684141115184 2023-07-29T13:31:50.954 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_cb from 4373 notify_id 386547056640 cookie 94684141116304 2023-07-29T13:31:50.954 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_cb from 4373 notify_id 390842023937 cookie 94684141115184 2023-07-29T13:31:50.955 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_cb from 4373 notify_id 390842023937 cookie 94684141116304 2023-07-29T13:31:50.955 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (16784 ms) 2023-07-29T13:31:50.955 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2023-07-29T13:31:50.955 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_cb from 4373 notify_id 408021893121 cookie 94684141118560 2023-07-29T13:31:50.956 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_cb from 4373 notify_id 412316860418 cookie 94684141118560 2023-07-29T13:31:50.956 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (7788 ms) 2023-07-29T13:31:50.956 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2023-07-29T13:31:50.957 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2023-07-29T13:31:50.957 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94684141099040 err -107 2023-07-29T13:31:50.957 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_cb from 4373 notify_id 442381631489 cookie 94684141099040 2023-07-29T13:31:50.957 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5252 ms) 2023-07-29T13:31:50.958 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2023-07-29T13:31:50.958 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2023-07-29T13:31:50.958 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94684141099040 err -107 2023-07-29T13:31:50.958 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (2019 ms) 2023-07-29T13:31:50.959 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (92885 ms total) 2023-07-29T13:31:50.959 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: 2023-07-29T13:31:50.959 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2023-07-29T13:31:50.960 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2023-07-29T13:31:50.960 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: watch_notify_test_cb 2023-07-29T13:31:50.960 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (6051 ms) 2023-07-29T13:31:50.960 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (6051 ms total) 2023-07-29T13:31:50.961 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: 2023-07-29T13:31:50.962 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [----------] Global test environment tear-down 2023-07-29T13:31:50.962 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (157135 ms total) 2023-07-29T13:31:50.962 INFO:tasks.workunit.client.0.smithi040.stdout: api_watch_notify: [ PASSED ] 11 tests. 2023-07-29T13:32:01.418 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1a00360 2023-07-29T13:32:01.419 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec0db1710 2023-07-29T13:32:01.419 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1a06ca0 2023-07-29T13:32:01.419 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec19c8ad0 2023-07-29T13:32:01.419 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1128a40 2023-07-29T13:32:01.419 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1129540 2023-07-29T13:32:01.419 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112a0d0 2023-07-29T13:32:01.419 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112ac40 2023-07-29T13:32:01.420 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112b370 2023-07-29T13:32:01.420 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112a880 2023-07-29T13:32:01.420 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112aa10 2023-07-29T13:32:01.420 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1aa96a0 2023-07-29T13:32:01.421 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1129d60 2023-07-29T13:32:01.421 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1aaa230 2023-07-29T13:32:01.421 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1aaad10 2023-07-29T13:32:01.421 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec11292f0 2023-07-29T13:32:01.422 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1aabd10 2023-07-29T13:32:01.422 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1aaaa40 2023-07-29T13:32:01.422 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1aac8d0 2023-07-29T13:32:01.422 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112b9d0 2023-07-29T13:32:01.423 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112c0f0 2023-07-29T13:32:01.423 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112cbd0 2023-07-29T13:32:01.423 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1aaba10 2023-07-29T13:32:01.424 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112d790 2023-07-29T13:32:01.424 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112e280 2023-07-29T13:32:01.424 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112c9b0 2023-07-29T13:32:01.425 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112f260 2023-07-29T13:32:01.425 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112e020 2023-07-29T13:32:01.425 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec11301d0 2023-07-29T13:32:01.426 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec112ef50 2023-07-29T13:32:01.426 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1130de0 2023-07-29T13:32:01.426 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: waiting 0x557ec1131940 2023-07-29T13:32:01.426 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: done. 2023-07-29T13:32:01.427 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (42280 ms) 2023-07-29T13:32:01.427 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (42280 ms total) 2023-07-29T13:32:01.427 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:32:01.428 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2023-07-29T13:32:01.428 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2023-07-29T13:32:01.428 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2023-07-29T13:32:01.428 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2023-07-29T13:32:01.428 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ OK ] LibRadosMisc.Exec (24079 ms) 2023-07-29T13:32:01.429 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2023-07-29T13:32:01.429 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (17950 ms) 2023-07-29T13:32:01.429 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2023-07-29T13:32:01.429 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (836 ms) 2023-07-29T13:32:01.430 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2023-07-29T13:32:01.430 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ OK ] LibRadosMisc.Applications (6279 ms) 2023-07-29T13:32:01.430 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2023-07-29T13:32:01.430 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2023-07-29T13:32:01.431 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2023-07-29T13:32:01.431 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2023-07-29T13:32:01.431 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2023-07-29T13:32:01.431 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (72302 ms) 2023-07-29T13:32:01.431 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [----------] 8 tests from LibRadosMisc (121447 ms total) 2023-07-29T13:32:01.431 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: 2023-07-29T13:32:01.432 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [----------] Global test environment tear-down 2023-07-29T13:32:01.432 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [==========] 11 tests from 4 test suites ran. (167701 ms total) 2023-07-29T13:32:01.432 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc: [ PASSED ] 11 tests. 2023-07-29T13:32:01.437 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: Running main() from gmock_main.cc 2023-07-29T13:32:01.437 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2023-07-29T13:32:01.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [----------] Global test environment set-up. 2023-07-29T13:32:01.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2023-07-29T13:32:01.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: seed 90899 2023-07-29T13:32:01.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2023-07-29T13:32:01.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (25066 ms) 2023-07-29T13:32:01.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2023-07-29T13:32:01.438 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (3167 ms) 2023-07-29T13:32:01.439 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2023-07-29T13:32:01.439 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (23744 ms) 2023-07-29T13:32:01.439 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2023-07-29T13:32:01.439 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (3361 ms) 2023-07-29T13:32:01.439 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2023-07-29T13:32:01.439 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (10599 ms) 2023-07-29T13:32:01.439 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2023-07-29T13:32:01.439 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (6070 ms) 2023-07-29T13:32:01.440 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2023-07-29T13:32:01.440 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (6901 ms) 2023-07-29T13:32:01.440 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2023-07-29T13:32:01.440 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (14832 ms) 2023-07-29T13:32:01.440 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (93740 ms total) 2023-07-29T13:32:01.440 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: 2023-07-29T13:32:01.440 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2023-07-29T13:32:01.441 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2023-07-29T13:32:01.441 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (6776 ms) 2023-07-29T13:32:01.441 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2023-07-29T13:32:01.441 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (196 ms) 2023-07-29T13:32:01.441 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2023-07-29T13:32:01.441 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (3301 ms) 2023-07-29T13:32:01.441 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2023-07-29T13:32:01.441 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1158 ms) 2023-07-29T13:32:01.442 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2023-07-29T13:32:01.442 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (265 ms) 2023-07-29T13:32:01.442 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2023-07-29T13:32:01.442 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (781 ms) 2023-07-29T13:32:01.442 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2023-07-29T13:32:01.442 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (891 ms) 2023-07-29T13:32:01.442 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2023-07-29T13:32:01.442 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (8962 ms) 2023-07-29T13:32:01.443 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (22330 ms total) 2023-07-29T13:32:01.443 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: 2023-07-29T13:32:01.443 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [----------] Global test environment tear-down 2023-07-29T13:32:01.443 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (167732 ms total) 2023-07-29T13:32:01.443 INFO:tasks.workunit.client.0.smithi040.stdout: api_lock_pp: [ PASSED ] 16 tests. 2023-07-29T13:32:02.579 INFO:tasks.workunit.client.0.smithi040.stdout:RoundTripPP2 (174 ms) 2023-07-29T13:32:02.579 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2023-07-29T13:32:02.580 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (10153 ms) 2023-07-29T13:32:02.580 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2023-07-29T13:32:02.580 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (335 ms) 2023-07-29T13:32:02.580 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2023-07-29T13:32:02.580 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (881 ms) 2023-07-29T13:32:02.580 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2023-07-29T13:32:02.581 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (639 ms) 2023-07-29T13:32:02.581 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2023-07-29T13:32:02.581 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (204 ms) 2023-07-29T13:32:02.581 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2023-07-29T13:32:02.581 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (192 ms) 2023-07-29T13:32:02.581 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2023-07-29T13:32:02.581 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (241 ms) 2023-07-29T13:32:02.582 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2023-07-29T13:32:02.582 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (507 ms) 2023-07-29T13:32:02.582 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2023-07-29T13:32:02.582 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (189 ms) 2023-07-29T13:32:02.582 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2023-07-29T13:32:02.582 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (174 ms) 2023-07-29T13:32:02.582 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2023-07-29T13:32:02.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (119 ms) 2023-07-29T13:32:02.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2023-07-29T13:32:02.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (847 ms) 2023-07-29T13:32:02.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (24115 ms total) 2023-07-29T13:32:02.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: 2023-07-29T13:32:02.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [----------] Global test environment tear-down 2023-07-29T13:32:02.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (168946 ms total) 2023-07-29T13:32:02.584 INFO:tasks.workunit.client.0.smithi040.stdout: api_io_pp: [ PASSED ] 37 tests. 2023-07-29T13:32:02.894 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: Running main() from gmock_main.cc 2023-07-29T13:32:02.894 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2023-07-29T13:32:02.894 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [----------] Global test environment set-up. 2023-07-29T13:32:02.894 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [----------] 14 tests from LibRadosIo 2023-07-29T13:32:02.894 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2023-07-29T13:32:02.894 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (26327 ms) 2023-07-29T13:32:02.895 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2023-07-29T13:32:02.895 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2023-07-29T13:32:02.895 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2023-07-29T13:32:02.895 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: no timeout :/ 2023-07-29T13:32:02.895 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: no timeout :/ 2023-07-29T13:32:02.895 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: no timeout :/ 2023-07-29T13:32:02.896 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: no timeout :/ 2023-07-29T13:32:02.896 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: no timeout :/ 2023-07-29T13:32:02.896 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (3499 ms) 2023-07-29T13:32:02.896 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2023-07-29T13:32:02.896 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (6914 ms) 2023-07-29T13:32:02.896 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2023-07-29T13:32:02.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.Checksum (12061 ms) 2023-07-29T13:32:02.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2023-07-29T13:32:02.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (7728 ms) 2023-07-29T13:32:02.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2023-07-29T13:32:02.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (202 ms) 2023-07-29T13:32:02.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2023-07-29T13:32:02.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (4866 ms) 2023-07-29T13:32:02.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2023-07-29T13:32:02.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2312 ms) 2023-07-29T13:32:02.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2023-07-29T13:32:02.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.TruncTest (2739 ms) 2023-07-29T13:32:02.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2023-07-29T13:32:02.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (19483 ms) 2023-07-29T13:32:02.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2023-07-29T13:32:02.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (11125 ms) 2023-07-29T13:32:02.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2023-07-29T13:32:02.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.RmXattr (567 ms) 2023-07-29T13:32:02.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2023-07-29T13:32:02.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIo.XattrIter (1164 ms) 2023-07-29T13:32:02.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [----------] 14 tests from LibRadosIo (98987 ms total) 2023-07-29T13:32:02.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: 2023-07-29T13:32:02.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2023-07-29T13:32:02.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2023-07-29T13:32:02.900 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (7067 ms) 2023-07-29T13:32:02.900 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2023-07-29T13:32:02.900 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (570 ms) 2023-07-29T13:32:02.900 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2023-07-29T13:32:02.901 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (296 ms) 2023-07-29T13:32:02.901 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2023-07-29T13:32:02.901 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (530 ms) 2023-07-29T13:32:02.901 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2023-07-29T13:32:02.901 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (1397 ms) 2023-07-29T13:32:02.901 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2023-07-29T13:32:02.901 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (10487 ms) 2023-07-29T13:32:02.901 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2023-07-29T13:32:02.902 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (1406 ms) 2023-07-29T13:32:02.902 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2023-07-29T13:32:02.902 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (186 ms) 2023-07-29T13:32:02.902 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2023-07-29T13:32:02.902 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (533 ms) 2023-07-29T13:32:02.902 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2023-07-29T13:32:02.902 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (307 ms) 2023-07-29T13:32:02.903 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [----------] 10 tests from LibRadosIoEC (22780 ms total) 2023-07-29T13:32:02.903 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: 2023-07-29T13:32:02.903 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [----------] Global test environment tear-down 2023-07-29T13:32:02.903 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [==========] 24 tests from 2 test suites ran. (169268 ms total) 2023-07-29T13:32:02.903 INFO:tasks.workunit.client.0.smithi040.stdout: api_io: [ PASSED ] 24 tests. 2023-07-29T13:32:04.525 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.525 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_9_[126418]: starting. 2023-07-29T13:32:04.526 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_9_[126418]: creating pool ceph_test_rados_list_parallel.smithi040-91409 2023-07-29T13:32:04.526 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_9_[126418]: created object 0... 2023-07-29T13:32:04.526 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_9_[126418]: created object 25... 2023-07-29T13:32:04.526 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_9_[126418]: created object 49... 2023-07-29T13:32:04.526 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_9_[126418]: finishing. 2023-07-29T13:32:04.526 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_9_[126418]: shutting down. 2023-07-29T13:32:04.526 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.527 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.527 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.527 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_10_[126419]: starting. 2023-07-29T13:32:04.527 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_10_[126419]: listing objects. 2023-07-29T13:32:04.527 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_10_[126419]: listed object 0... 2023-07-29T13:32:04.528 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_10_[126419]: listed object 25... 2023-07-29T13:32:04.528 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_10_[126419]: listed object 50... 2023-07-29T13:32:04.528 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_10_[126419]: listed object 75... 2023-07-29T13:32:04.528 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_10_[126419]: listed object 100... 2023-07-29T13:32:04.528 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_10_[126419]: saw 104 objects 2023-07-29T13:32:04.528 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_10_[126419]: shutting down. 2023-07-29T13:32:04.528 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.529 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.529 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.529 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_12_[126421]: starting. 2023-07-29T13:32:04.529 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_12_[126421]: added 25 objects... 2023-07-29T13:32:04.529 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_12_[126421]: added half of the objects 2023-07-29T13:32:04.529 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_12_[126421]: added 50 objects... 2023-07-29T13:32:04.529 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_12_[126421]: added 50 objects 2023-07-29T13:32:04.529 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_12_[126421]: shutting down. 2023-07-29T13:32:04.530 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.530 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.530 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.530 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_11_[126420]: starting. 2023-07-29T13:32:04.530 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_11_[126420]: added 25 objects... 2023-07-29T13:32:04.530 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_11_[126420]: added half of the objects 2023-07-29T13:32:04.530 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_11_[126420]: added 50 objects... 2023-07-29T13:32:04.531 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_11_[126420]: added 50 objects 2023-07-29T13:32:04.531 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_11_[126420]: shutting down. 2023-07-29T13:32:04.531 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.531 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.531 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.531 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_13_[126422]: starting. 2023-07-29T13:32:04.531 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_13_[126422]: removed 25 objects... 2023-07-29T13:32:04.531 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_13_[126422]: removed half of the objects 2023-07-29T13:32:04.532 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_13_[126422]: removed 50 objects... 2023-07-29T13:32:04.532 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_13_[126422]: removed 50 objects 2023-07-29T13:32:04.532 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_13_[126422]: shutting down. 2023-07-29T13:32:04.532 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.532 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.532 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.533 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.533 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_14_[147125]: starting. 2023-07-29T13:32:04.533 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_14_[147125]: creating pool ceph_test_rados_list_parallel.smithi040-91409 2023-07-29T13:32:04.533 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_14_[147125]: created object 0... 2023-07-29T13:32:04.533 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_14_[147125]: created object 25... 2023-07-29T13:32:04.533 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_14_[147125]: created object 49... 2023-07-29T13:32:04.533 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_14_[147125]: finishing. 2023-07-29T13:32:04.533 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_14_[147125]: shutting down. 2023-07-29T13:32:04.534 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.534 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.534 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.534 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:04.534 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_15_[147126]: starting. 2023-07-29T13:32:04.896 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_15_[147 api_c_write_operations: Running main() from gmock_main.cc 2023-07-29T13:32:04.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2023-07-29T13:32:04.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [----------] Global test environment set-up. 2023-07-29T13:32:04.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2023-07-29T13:32:04.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2023-07-29T13:32:04.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2023-07-29T13:32:04.897 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2023-07-29T13:32:04.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: 2023-07-29T13:32:04.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2023-07-29T13:32:04.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2023-07-29T13:32:04.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (16046 ms) 2023-07-29T13:32:04.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2023-07-29T13:32:04.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (22759 ms) 2023-07-29T13:32:04.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2023-07-29T13:32:04.898 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (32654 ms) 2023-07-29T13:32:04.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2023-07-29T13:32:04.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (32197 ms) 2023-07-29T13:32:04.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2023-07-29T13:32:04.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (27749 ms) 2023-07-29T13:32:04.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2023-07-29T13:32:04.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (32497 ms) 2023-07-29T13:32:04.899 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2023-07-29T13:32:04.900 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (7073 ms) 2023-07-29T13:32:04.900 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (170975 ms total) 2023-07-29T13:32:04.900 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: 2023-07-29T13:32:04.900 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [----------] Global test environment tear-down 2023-07-29T13:32:04.900 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (170976 ms total) 2023-07-29T13:32:04.900 INFO:tasks.workunit.client.0.smithi040.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2023-07-29T13:32:19.095 INFO:tasks.workunit.client.0.smithi040.stdout:126]: listing objects. 2023-07-29T13:32:19.096 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_15_[147126]: listed object 0... 2023-07-29T13:32:19.096 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_15_[147126]: listed object 25... 2023-07-29T13:32:19.096 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_15_[147126]: listed object 50... 2023-07-29T13:32:19.096 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_15_[147126]: listed object 75... 2023-07-29T13:32:19.096 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_15_[147126]: listed object 100... 2023-07-29T13:32:19.097 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_15_[147126]: listed object 125... 2023-07-29T13:32:19.097 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_15_[147126]: saw 150 objects 2023-07-29T13:32:19.097 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_15_[147126]: shutting down. 2023-07-29T13:32:19.097 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:19.097 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:19.097 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:19.098 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:19.098 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_16_[147127]: starting. 2023-07-29T13:32:19.098 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_16_[147127]: added 25 objects... 2023-07-29T13:32:19.098 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_16_[147127]: added half of the objects 2023-07-29T13:32:19.098 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_16_[147127]: added 50 objects... 2023-07-29T13:32:19.098 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_16_[147127]: added 50 objects 2023-07-29T13:32:19.098 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: process_16_[147127]: shutting down. 2023-07-29T13:32:19.098 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:19.099 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:19.099 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:19.099 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:19.099 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******************************* 2023-07-29T13:32:19.099 INFO:tasks.workunit.client.0.smithi040.stdout: list_parallel: ******* SUCCESS ********** 2023-07-29T13:32:22.914 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:05:18.248 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:22.919 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:05:18.253 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:22.986 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:05:18.271 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:22.998 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:05:18.282 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:23.012 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:05:18.346 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:23.017 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:05:18.351 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:23.172 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:05:18.506 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:23.199 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:05:18.483 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:23.260 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:05:18.594 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:41.158 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: Running main() from gmock_main.cc 2023-07-29T13:32:41.159 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2023-07-29T13:32:41.159 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [----------] Global test environment set-up. 2023-07-29T13:32:41.159 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2023-07-29T13:32:41.159 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2023-07-29T13:32:41.159 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (30073 ms) 2023-07-29T13:32:41.159 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2023-07-29T13:32:41.160 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (23205 ms) 2023-07-29T13:32:41.160 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2023-07-29T13:32:41.160 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (5943 ms) 2023-07-29T13:32:41.160 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2023-07-29T13:32:41.160 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (3221 ms) 2023-07-29T13:32:41.160 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (62442 ms total) 2023-07-29T13:32:41.160 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: 2023-07-29T13:32:41.161 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2023-07-29T13:32:41.161 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2023-07-29T13:32:41.161 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (16634 ms) 2023-07-29T13:32:41.161 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2023-07-29T13:32:41.161 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (6831 ms) 2023-07-29T13:32:41.161 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2023-07-29T13:32:41.161 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (38428 ms) 2023-07-29T13:32:41.161 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (61893 ms total) 2023-07-29T13:32:41.162 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: 2023-07-29T13:32:41.162 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2023-07-29T13:32:41.162 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2023-07-29T13:32:41.162 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (6189 ms) 2023-07-29T13:32:41.162 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2023-07-29T13:32:41.162 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (3209 ms) 2023-07-29T13:32:41.162 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2023-07-29T13:32:41.162 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (4511 ms) 2023-07-29T13:32:41.163 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2023-07-29T13:32:41.163 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (3977 ms) 2023-07-29T13:32:41.163 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (17887 ms total) 2023-07-29T13:32:41.163 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: 2023-07-29T13:32:41.163 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2023-07-29T13:32:41.163 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2023-07-29T13:32:41.163 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (6614 ms) 2023-07-29T13:32:41.163 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2023-07-29T13:32:41.164 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (7165 ms) 2023-07-29T13:32:41.164 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (13779 ms total) 2023-07-29T13:32:41.164 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: 2023-07-29T13:32:41.164 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [----------] Global test environment tear-down 2023-07-29T13:32:41.164 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (207383 ms total) 2023-07-29T13:32:41.165 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots: [ PASSED ] 13 tests. 2023-07-29T13:32:43.028 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:05:38.362 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:43.095 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:05:38.429 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:43.139 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:05:38.473 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:43.159 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:05:38.443 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:43.164 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:05:38.448 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:43.172 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:05:38.506 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:43.178 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:05:38.512 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:43.238 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:05:38.523 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:43.243 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:05:38.577 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:32:56.689 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2023-07-29T13:32:56.689 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [==========] Running 19 tests from 4 test suites. 2023-07-29T13:32:56.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2023-07-29T13:32:56.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2023-07-29T13:32:56.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: seed 91038 2023-07-29T13:32:56.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2023-07-29T13:32:56.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (28725 ms) 2023-07-29T13:32:56.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2023-07-29T13:32:56.690 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (4436 ms) 2023-07-29T13:32:56.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2023-07-29T13:32:56.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (9935 ms) 2023-07-29T13:32:56.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2023-07-29T13:32:56.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (9890 ms) 2023-07-29T13:32:56.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2023-07-29T13:32:56.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (11423 ms) 2023-07-29T13:32:56.691 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (64409 ms total) 2023-07-29T13:32:56.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: 2023-07-29T13:32:56.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2023-07-29T13:32:56.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2023-07-29T13:32:56.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (42940 ms) 2023-07-29T13:32:56.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2023-07-29T13:32:56.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (37431 ms) 2023-07-29T13:32:56.692 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2023-07-29T13:32:56.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (20157 ms) 2023-07-29T13:32:56.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2023-07-29T13:32:56.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (5424 ms) 2023-07-29T13:32:56.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2023-07-29T13:32:56.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (3199 ms) 2023-07-29T13:32:56.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2023-07-29T13:32:56.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.1.3-11-g0484097b/rpm/el9/BUILD/ceph-18.1.3-11-g0484097b/src/test/librados/snapshots_cxx.cc:463: Skipped 2023-07-29T13:32:56.693 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: 2023-07-29T13:32:56.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2023-07-29T13:32:56.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2023-07-29T13:32:56.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi040-91038-7 2023-07-29T13:32:56.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: waiting for snaps to purge 2023-07-29T13:32:56.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (18829 ms) 2023-07-29T13:32:56.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (127980 ms total) 2023-07-29T13:32:56.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: 2023-07-29T13:32:56.694 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2023-07-29T13:32:56.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2023-07-29T13:32:56.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (4095 ms) 2023-07-29T13:32:56.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2023-07-29T13:32:56.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2104 ms) 2023-07-29T13:32:56.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2023-07-29T13:32:56.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (3474 ms) 2023-07-29T13:32:56.695 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2023-07-29T13:32:56.696 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2432 ms) 2023-07-29T13:32:56.696 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (12105 ms total) 2023-07-29T13:32:56.696 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: 2023-07-29T13:32:56.696 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2023-07-29T13:33:03.070 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:05:58.404 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:03.153 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:05:58.487 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:03.157 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:05:58.491 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:03.160 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:05:58.495 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:03.215 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:05:58.549 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:03.230 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:05:58.564 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:03.238 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:05:58.522 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:03.246 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:05:58.530 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:03.300 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:05:58.584 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:15.383 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2023-07-29T13:33:15.383 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (6328 ms) 2023-07-29T13:33:15.383 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2023-07-29T13:33:15.383 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (5205 ms) 2023-07-29T13:33:15.383 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2023-07-29T13:33:15.383 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4695 ms) 2023-07-29T13:33:15.384 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (16228 ms total) 2023-07-29T13:33:15.384 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: 2023-07-29T13:33:15.384 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2023-07-29T13:33:15.385 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [==========] 19 tests from 4 test suites ran. (241618 ms total) 2023-07-29T13:33:15.385 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ PASSED ] 18 tests. 2023-07-29T13:33:15.385 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2023-07-29T13:33:15.386 INFO:tasks.workunit.client.0.smithi040.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2023-07-29T13:33:16.452 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: Running main() from gmock_main.cc 2023-07-29T13:33:16.452 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2023-07-29T13:33:16.452 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [----------] Global test environment set-up. 2023-07-29T13:33:16.453 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [----------] 26 tests from LibRadosAio 2023-07-29T13:33:16.453 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2023-07-29T13:33:16.453 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.TooBig (7215 ms) 2023-07-29T13:33:16.453 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2023-07-29T13:33:16.453 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (25198 ms) 2023-07-29T13:33:16.454 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2023-07-29T13:33:16.454 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (8396 ms) 2023-07-29T13:33:16.454 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2023-07-29T13:33:16.454 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (38179 ms) 2023-07-29T13:33:16.454 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2023-07-29T13:33:16.455 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (29971 ms) 2023-07-29T13:33:16.455 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2023-07-29T13:33:16.455 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (26323 ms) 2023-07-29T13:33:16.455 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2023-07-29T13:33:16.455 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (25983 ms) 2023-07-29T13:33:16.455 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2023-07-29T13:33:16.455 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (10010 ms) 2023-07-29T13:33:16.455 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2023-07-29T13:33:16.456 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3225 ms) 2023-07-29T13:33:16.456 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2023-07-29T13:33:16.456 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (4452 ms) 2023-07-29T13:33:16.456 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2023-07-29T13:33:16.456 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (4336 ms) 2023-07-29T13:33:16.456 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2023-07-29T13:33:16.456 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (4213 ms) 2023-07-29T13:33:16.457 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2023-07-29T13:33:16.457 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3904 ms) 2023-07-29T13:33:16.457 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2023-07-29T13:33:16.457 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3857 ms) 2023-07-29T13:33:16.457 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2023-07-29T13:33:16.457 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.Flush (3671 ms) 2023-07-29T13:33:16.457 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2023-07-29T13:33:16.457 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (4530 ms) 2023-07-29T13:33:16.458 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2023-07-29T13:33:16.458 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (4065 ms) 2023-07-29T13:33:16.458 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2023-07-29T13:33:16.458 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3308 ms) 2023-07-29T13:33:16.458 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2023-07-29T13:33:16.458 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3392 ms) 2023-07-29T13:33:16.458 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2023-07-29T13:33:16.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (4543 ms) 2023-07-29T13:33:16.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2023-07-29T13:33:16.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (4294 ms) 2023-07-29T13:33:16.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2023-07-29T13:33:16.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3488 ms) 2023-07-29T13:33:16.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2023-07-29T13:33:16.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (4520 ms) 2023-07-29T13:33:16.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2023-07-29T13:33:16.460 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3513 ms) 2023-07-29T13:33:16.460 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2023-07-29T13:33:16.460 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (4700 ms) 2023-07-29T13:33:16.460 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2023-07-29T13:33:16.460 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3543 ms) 2023-07-29T13:33:16.460 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [----------] 26 tests from LibRadosAio (242831 ms total) 2023-07-29T13:33:16.460 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: 2023-07-29T13:33:16.461 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2023-07-29T13:33:23.354 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:06:18.688 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:23.405 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:06:18.690 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:23.410 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:06:18.694 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:23.414 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:06:18.748 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:23.468 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:06:18.753 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:23.478 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:06:18.813 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:23.482 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:06:18.816 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:23.505 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:06:18.840 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:23.552 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:06:18.886 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:36.600 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.Simple api_aio_pp: Running main() from gmock_main.cc 2023-07-29T13:33:36.600 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2023-07-29T13:33:36.600 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [----------] Global test environment set-up. 2023-07-29T13:33:36.600 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2023-07-29T13:33:36.600 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2023-07-29T13:33:36.600 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (4546 ms) 2023-07-29T13:33:36.601 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2023-07-29T13:33:36.601 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (61334 ms) 2023-07-29T13:33:36.601 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2023-07-29T13:33:36.601 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (42587 ms) 2023-07-29T13:33:36.601 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2023-07-29T13:33:36.601 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (26804 ms) 2023-07-29T13:33:36.601 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2023-07-29T13:33:36.602 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (23164 ms) 2023-07-29T13:33:36.602 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2023-07-29T13:33:36.602 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (11814 ms) 2023-07-29T13:33:36.602 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2023-07-29T13:33:36.602 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (4241 ms) 2023-07-29T13:33:36.602 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2023-07-29T13:33:36.602 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (4452 ms) 2023-07-29T13:33:36.602 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2023-07-29T13:33:36.603 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (5203 ms) 2023-07-29T13:33:36.603 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2023-07-29T13:33:36.603 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (4356 ms) 2023-07-29T13:33:36.603 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2023-07-29T13:33:36.603 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2907 ms) 2023-07-29T13:33:36.603 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2023-07-29T13:33:36.603 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3850 ms) 2023-07-29T13:33:36.603 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2023-07-29T13:33:36.604 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3700 ms) 2023-07-29T13:33:36.604 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2023-07-29T13:33:36.604 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (4474 ms) 2023-07-29T13:33:36.604 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2023-07-29T13:33:36.604 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (4074 ms) 2023-07-29T13:33:36.604 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2023-07-29T13:33:36.604 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3239 ms) 2023-07-29T13:33:36.604 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2023-07-29T13:33:36.605 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3478 ms) 2023-07-29T13:33:36.605 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2023-07-29T13:33:36.605 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (4552 ms) 2023-07-29T13:33:36.605 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2023-07-29T13:33:36.605 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3133 ms) 2023-07-29T13:33:36.605 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2023-07-29T13:33:36.605 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3629 ms) 2023-07-29T13:33:36.605 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2023-07-29T13:33:36.606 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3259 ms) 2023-07-29T13:33:36.606 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2023-07-29T13:33:36.606 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (12002 ms) 2023-07-29T13:33:36.606 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2023-07-29T13:33:36.606 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3005 ms) 2023-07-29T13:33:36.606 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2023-07-29T13:33:36.606 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3214 ms) 2023-07-29T13:33:36.606 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2023-07-29T13:33:36.607 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3823 ms) 2023-07-29T13:33:36.607 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2023-07-29T13:33:36.607 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3067 ms) 2023-07-29T13:33:36.607 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2023-07-29T13:33:43.244 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:06:38.579 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:43.292 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:06:38.576 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:43.293 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:06:38.577 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:43.294 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:06:38.629 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:43.296 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:06:38.630 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:43.298 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:06:38.632 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:43.338 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:06:38.622 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:43.345 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:06:38.680 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:33:43.355 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:06:38.689 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:03.345 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:06:58.679 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:03.347 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:06:58.682 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:03.386 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:06:58.720 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:03.396 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:06:58.680 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:03.399 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:06:58.684 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:03.409 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:06:58.743 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:03.413 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:06:58.747 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:03.423 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:06:58.708 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:03.438 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:06:58.773 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:03.531 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:34:03.528+0000 12a61640 -1 osd.6 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20020.0:1 128.6s0 128.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e218) 2023-07-29T13:34:04.506 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:34:04.503+0000 12a61640 -1 osd.6 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20020.0:1 128.6s0 128.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e218) 2023-07-29T13:34:05.478 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:34:05.474+0000 12a61640 -1 osd.6 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20020.0:1 128.6s0 128.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e218) 2023-07-29T13:34:06.458 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:34:06.454+0000 12a61640 -1 osd.6 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20020.0:1 128.6s0 128.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e218) 2023-07-29T13:34:07.507 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:34:07.505+0000 12a61640 -1 osd.6 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20020.0:1 128.6s0 128.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e218) 2023-07-29T13:34:08.493 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:34:08.490+0000 12a61640 -1 osd.6 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20020.0:1 128.6s0 128.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e218) 2023-07-29T13:34:09.504 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:34:09.501+0000 12a61640 -1 osd.6 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20020.0:1 128.6s0 128.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e218) 2023-07-29T13:34:10.491 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:34:10.488+0000 12a61640 -1 osd.6 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20020.0:1 128.6s0 128.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e218) 2023-07-29T13:34:11.429 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:34:11.422+0000 12a61640 -1 osd.0 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4340.0:2079 18.17 18:e86740d7:test-rados-api-smithi040-90908-17::big2.copy:head [copy-from ver 0 in=75b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e222) 2023-07-29T13:34:11.510 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:34:11.507+0000 12a61640 -1 osd.6 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20020.0:1 128.6s0 128.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e218) 2023-07-29T13:34:12.392 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:34:12.390+0000 12a61640 -1 osd.0 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4340.0:2079 18.17 18:e86740d7:test-rados-api-smithi040-90908-17::big2.copy:head [copy-from ver 0 in=75b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e222) 2023-07-29T13:34:12.497 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:34:12.493+0000 12a61640 -1 osd.6 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20020.0:1 128.6s0 128.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e218) 2023-07-29T13:34:23.495 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:18.829 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:23.500 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:18.834 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:23.542 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:18.876 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:23.549 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:18.883 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:23.552 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:18.836 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:23.553 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:18.888 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:23.554 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:18.838 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:23.590 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:18.874 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:23.595 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:18.929 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:27.908 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:23.242 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:27.910 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:23.245 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:27.940 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:23.274 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:27.958 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:23.243 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:27.964 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:23.248 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:27.978 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:23.312 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:27.980 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:23.314 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:27.983 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:23.268 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:27.999 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:23.333 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:43.699 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:39.034 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:43.702 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:39.036 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:43.747 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:39.081 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:43.760 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:39.095 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:43.762 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:39.097 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:43.763 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:39.048 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:43.768 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:39.052 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:43.807 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:39.142 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:43.816 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:39.100 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:34:49.450 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRad api_list: : cursor()=8:de5d7c5f:::12:head expected=8:de5d7c5f:::12:head 2023-07-29T13:34:49.451 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:de5d7c5f:::12:head -> 12 2023-07-29T13:34:49.451 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=12 expected=12 2023-07-29T13:34:49.451 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:d83876eb:::4:head 2023-07-29T13:34:49.451 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:d83876eb:::4:head expected=8:d83876eb:::4:head 2023-07-29T13:34:49.451 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:d83876eb:::4:head -> 4 2023-07-29T13:34:49.451 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=4 expected=4 2023-07-29T13:34:49.451 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:cfc208b3:::3:head 2023-07-29T13:34:49.452 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:cfc208b3:::3:head expected=8:cfc208b3:::3:head 2023-07-29T13:34:49.452 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:cfc208b3:::3:head -> 3 2023-07-29T13:34:49.452 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=3 expected=3 2023-07-29T13:34:49.452 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:c4fdafeb:::6:head 2023-07-29T13:34:49.452 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:c4fdafeb:::6:head expected=8:c4fdafeb:::6:head 2023-07-29T13:34:49.452 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:c4fdafeb:::6:head -> 6 2023-07-29T13:34:49.452 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=6 expected=6 2023-07-29T13:34:49.453 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:bd63b0f1:::8:head 2023-07-29T13:34:49.453 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:bd63b0f1:::8:head expected=8:bd63b0f1:::8:head 2023-07-29T13:34:49.453 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:bd63b0f1:::8:head -> 8 2023-07-29T13:34:49.453 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=8 expected=8 2023-07-29T13:34:49.454 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:b29083e3:::5:head 2023-07-29T13:34:49.454 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:b29083e3:::5:head expected=8:b29083e3:::5:head 2023-07-29T13:34:49.454 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:b29083e3:::5:head -> 5 2023-07-29T13:34:49.454 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=5 expected=5 2023-07-29T13:34:49.455 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:89d3ae78:::11:head 2023-07-29T13:34:49.455 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:89d3ae78:::11:head expected=8:89d3ae78:::11:head 2023-07-29T13:34:49.455 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:89d3ae78:::11:head -> 11 2023-07-29T13:34:49.456 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=11 expected=11 2023-07-29T13:34:49.456 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : seek to 8:02547ec2:::1:head 2023-07-29T13:34:49.456 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : cursor()=8:02547ec2:::1:head expected=8:02547ec2:::1:head 2023-07-29T13:34:49.457 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: > 8:02547ec2:::1:head -> 1 2023-07-29T13:34:49.457 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: : entry=1 expected=1 2023-07-29T13:34:49.457 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (10099 ms) 2023-07-29T13:34:49.458 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2023-07-29T13:34:49.458 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (94654 ms) 2023-07-29T13:34:49.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2023-07-29T13:34:49.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: split 0/5 -> MIN 8:33333333::::head 2023-07-29T13:34:49.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: split 1/5 -> 8:33333333::::head 8:66666666::::head 2023-07-29T13:34:49.459 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: split 2/5 -> 8:66666666::::head 8:99999999::::head 2023-07-29T13:34:49.460 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: split 3/5 -> 8:99999999::::head 8:cccccccc::::head 2023-07-29T13:34:49.460 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: split 4/5 -> 8:cccccccc::::head MAX 2023-07-29T13:34:49.460 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (100377 ms) 2023-07-29T13:34:49.460 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [----------] 7 tests from LibRadosList (319579 ms total) 2023-07-29T13:34:49.461 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 2023-07-29T13:34:49.461 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [----------] 3 tests from LibRadosListEC 2023-07-29T13:34:49.461 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2023-07-29T13:34:49.461 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1892 ms) 2023-07-29T13:34:49.462 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2023-07-29T13:34:49.462 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: myset foo1,foo2,foo3 2023-07-29T13:34:49.462 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo1 2023-07-29T13:34:49.462 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo2 2023-07-29T13:34:49.463 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo3 2023-07-29T13:34:49.463 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: myset foo1,foo4,foo5 2023-07-29T13:34:49.463 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo4 2023-07-29T13:34:49.463 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo5 2023-07-29T13:34:49.463 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo1 2023-07-29T13:34:49.463 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: myset foo6,foo7 2023-07-29T13:34:49.463 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo7 2023-07-29T13:34:49.464 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: foo6 2023-07-29T13:34:49.464 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2023-07-29T13:34:49.464 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: ns1:foo4 2023-07-29T13:34:49.464 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: ns1:foo5 2023-07-29T13:34:49.464 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: ns2:foo7 2023-07-29T13:34:49.464 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: ns2:foo6 2023-07-29T13:34:49.464 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: ns1:foo1 2023-07-29T13:34:49.465 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: :foo1 2023-07-29T13:34:49.465 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: :foo2 2023-07-29T13:34:49.465 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: :foo3 2023-07-29T13:34:49.465 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (2657 ms) 2023-07-29T13:34:49.465 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2023-07-29T13:34:49.465 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 1 0 2023-07-29T13:34:49.465 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 10 0 2023-07-29T13:34:49.465 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 13 0 2023-07-29T13:34:53.141 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2023-07-29T13:34:53.142 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] Global test environment set-up. 2023-07-29T13:34:53.142 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2023-07-29T13:34:53.142 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2023-07-29T13:34:53.142 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2023-07-29T13:34:53.142 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2023-07-29T13:34:53.143 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: 2023-07-29T13:34:53.143 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2023-07-29T13:34:53.143 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: seed 90908 2023-07-29T13:34:53.143 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2023-07-29T13:34:53.143 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (151 ms) 2023-07-29T13:34:53.143 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2023-07-29T13:34:53.143 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (35882 ms) 2023-07-29T13:34:53.143 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2023-07-29T13:34:53.144 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (26282 ms) 2023-07-29T13:34:53.144 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2023-07-29T13:34:53.144 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (9250 ms) 2023-07-29T13:34:53.144 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2023-07-29T13:34:53.144 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (7276 ms) 2023-07-29T13:34:53.144 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2023-07-29T13:34:53.144 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2502 ms) 2023-07-29T13:34:53.145 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2023-07-29T13:34:53.145 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (1118 ms) 2023-07-29T13:34:53.145 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2023-07-29T13:34:53.145 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (1298 ms) 2023-07-29T13:34:53.145 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2023-07-29T13:34:53.145 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (1389 ms) 2023-07-29T13:34:53.145 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2023-07-29T13:34:53.145 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (594 ms) 2023-07-29T13:34:53.146 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2023-07-29T13:34:53.146 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (1195 ms) 2023-07-29T13:34:53.146 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2023-07-29T13:34:53.146 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (1255 ms) 2023-07-29T13:34:53.146 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2023-07-29T13:34:53.146 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (7228 ms) 2023-07-29T13:34:53.146 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2023-07-29T13:34:53.146 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: osd_max_attr_size = 0 2023-07-29T13:34:53.147 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2023-07-29T13:34:53.147 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (108967 ms) 2023-07-29T13:34:53.147 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2023-07-29T13:34:53.147 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (13472 ms) 2023-07-29T13:34:53.147 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2023-07-29T13:34:53.147 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: waiting for initial deep scrubs... 2023-07-29T13:34:53.147 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: done waiting, doing copies 2023-07-29T13:34:53.147 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: waiting for final deep scrubs... 2023-07-29T13:34:53.148 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: done waiting 2023-07-29T13:34:53.148 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (109029 ms) 2023-07-29T13:34:53.148 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2023-07-29T13:34:53.148 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (1003 ms) 2023-07-29T13:34:53.148 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2023-07-29T13:34:53.148 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (106 ms) 2023-07-29T13:34:53.148 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2023-07-29T13:34:53.149 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (5597 ms) 2023-07-29T13:34:53.149 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2023-07-29T13:34:53.149 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2023-07-29T13:34:53.149 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2023-07-29T13:34:53.149 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2023-07-29T13:34:53.149 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2023-07-29T13:34:53.149 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2023-07-29T13:34:56.050 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp api_list: 7 0 2023-07-29T13:34:56.050 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 14 0 2023-07-29T13:34:56.051 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 0 0 2023-07-29T13:34:56.051 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 15 0 2023-07-29T13:34:56.051 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 11 0 2023-07-29T13:34:56.051 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 5 0 2023-07-29T13:34:56.051 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 8 0 2023-07-29T13:34:56.051 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 6 0 2023-07-29T13:34:56.051 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 3 0 2023-07-29T13:34:56.052 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 4 0 2023-07-29T13:34:56.052 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 12 0 2023-07-29T13:34:56.052 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 9 0 2023-07-29T13:34:56.052 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 2 0 2023-07-29T13:34:56.052 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2023-07-29T13:34:56.052 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (4123 ms) 2023-07-29T13:34:56.052 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [----------] 3 tests from LibRadosListEC (8672 ms total) 2023-07-29T13:34:56.052 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 2023-07-29T13:34:56.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [----------] 1 test from LibRadosListNP 2023-07-29T13:34:56.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2023-07-29T13:34:56.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (2933 ms) 2023-07-29T13:34:56.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [----------] 1 test from LibRadosListNP (2933 ms total) 2023-07-29T13:34:56.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: 2023-07-29T13:34:56.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [----------] Global test environment tear-down 2023-07-29T13:34:56.053 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [==========] 11 tests from 3 test suites ran. (342353 ms total) 2023-07-29T13:34:56.054 INFO:tasks.workunit.client.0.smithi040.stdout: api_list: [ PASSED ] 11 tests. 2023-07-29T13:35:03.931 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:59.265 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:03.933 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:59.267 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:03.986 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:59.271 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:03.989 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:07:59.324 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:03.991 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:59.275 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:03.996 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:59.330 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:03.998 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:59.332 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:04.043 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:07:59.327 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:04.076 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:07:59.410 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:23.840 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:08:19.175 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:23.895 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:08:19.179 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:23.897 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:08:19.182 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:23.903 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:08:19.238 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:23.956 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:08:19.290 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:23.964 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:08:19.298 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:23.966 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:08:19.300 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:23.968 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:08:19.252 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:24.015 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:08:19.349 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:45.727 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:45.720+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:46.773 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:46.771+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:47.773 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:47.770+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:48.746 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:48.744+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:48.961 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:08:44.295 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:48.967 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:08:44.301 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:48.993 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:08:44.327 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:49.012 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:08:44.296 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:49.016 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:08:44.300 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:49.017 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:08:44.351 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:49.020 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:08:44.354 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:49.042 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:08:44.327 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:49.045 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:08:44.380 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:35:49.799 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:49.797+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:50.784 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:50.781+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:51.762 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:51.760+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:52.809 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:52.807+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:53.791 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:53.789+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:54.781 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:54.778+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:55.791 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:55.788+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:56.781 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:56.779+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:57.746 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:57.743+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:58.739 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:58.737+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:35:59.726 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:35:59.723+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:00.705 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:00.702+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:01.729 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:01.726+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:02.759 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:02.756+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:03.755 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:03.753+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:04.754 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:04.751+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:05.707 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:05.704+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:06.705 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:06.702+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:07.671 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:07.668+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:08.643 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:08.641+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:09.020 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:04.354 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:09.026 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:04.360 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:09.048 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:04.383 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:09.070 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:04.354 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:09.074 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:04.358 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:09.076 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:04.410 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:09.080 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:04.415 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:09.102 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:04.437 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:09.105 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:04.389 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:09.633 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:09.630+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:10.672 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:10.669+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:11.627 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:11.624+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:12.621 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:12.619+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:13.578 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:13.575+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:14.595 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:14.592+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:15.618 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:15.616+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:16.628 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:16.625+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:17.621 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:17.619+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:18.644 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:18.640+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:19.629 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:19.627+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:20.622 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:20.619+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:21.632 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:21.630+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:22.686 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:22.683+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:23.715 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:23.713+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:24.669 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:24.667+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:25.632 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:25.630+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:26.448 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:36:26.383+0000 12a61640 -1 osd.7 292 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14697.0:2 159.12 159.620ee612 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e292) 2023-07-29T13:36:26.602 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:26.599+0000 12a61640 -1 osd.2 300 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:27.272 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:36:27.226+0000 12a61640 -1 osd.6 290 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:976 98.b 98:d76b5f17:test-rados-api-smithi040-90886-14::foo7:6 [read 0~1] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e289) 2023-07-29T13:36:27.547 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:36:27.538+0000 12a61640 -1 osd.7 301 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14697.0:2 159.12 159:48677046:test-rados-api-smithi040-90908-31::foo:head [write 0~4096 in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292) 2023-07-29T13:36:27.586 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:27.583+0000 12a61640 -1 osd.2 301 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:28.405 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:36:28.366+0000 12a61640 -1 osd.6 301 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:976 98.b 98:d76b5f17:test-rados-api-smithi040-90886-14::foo7:6 [read 0~1] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e289) 2023-07-29T13:36:28.556 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:36:28.554+0000 12a61640 -1 osd.2 302 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.16232.0:1 158.6s0 158:602f83fe:::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e290) 2023-07-29T13:36:29.078 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:24.412 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:29.081 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:24.415 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:29.118 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:24.452 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:29.129 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:24.413 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:29.133 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:24.417 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:29.149 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:24.483 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:29.156 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:24.490 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:29.158 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:24.442 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:29.193 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:24.527 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:35.361 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:30.695 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:35.371 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:30.705 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:35.405 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:30.739 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:35.435 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:30.769 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:35.437 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:30.717 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:35.437 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:30.719 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:35.445 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:30.779 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:35.470 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:30.755 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:35.480 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:30.814 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:40.468 INFO:tasks.workunit.client.0.smithi040.stdout:: [----------] 22 tests from LibRadosMiscPP (333594 ms total) 2023-07-29T13:36:40.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: 2023-07-29T13:36:40.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2023-07-29T13:36:40.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2023-07-29T13:36:40.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (8930 ms) 2023-07-29T13:36:40.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (8930 ms total) 2023-07-29T13:36:40.469 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: 2023-07-29T13:36:40.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2023-07-29T13:36:40.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2023-07-29T13:36:40.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (718 ms) 2023-07-29T13:36:40.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2023-07-29T13:36:40.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (58 ms) 2023-07-29T13:36:40.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (776 ms total) 2023-07-29T13:36:40.470 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: 2023-07-29T13:36:40.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2023-07-29T13:36:40.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2023-07-29T13:36:40.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (70570 ms) 2023-07-29T13:36:40.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2023-07-29T13:36:40.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (1423 ms) 2023-07-29T13:36:40.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (71994 ms total) 2023-07-29T13:36:40.471 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: 2023-07-29T13:36:40.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2023-07-29T13:36:40.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2023-07-29T13:36:40.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (597 ms) 2023-07-29T13:36:40.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2023-07-29T13:36:40.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (78 ms) 2023-07-29T13:36:40.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (675 ms total) 2023-07-29T13:36:40.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: 2023-07-29T13:36:40.472 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2023-07-29T13:36:40.473 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2023-07-29T13:36:40.473 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1849 ms) 2023-07-29T13:36:40.473 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1849 ms total) 2023-07-29T13:36:40.473 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: 2023-07-29T13:36:40.473 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [----------] Global test environment tear-down 2023-07-29T13:36:40.473 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (446739 ms total) 2023-07-29T13:36:40.473 INFO:tasks.workunit.client.0.smithi040.stdout: api_misc_pp: [ PASSED ] 31 tests. 2023-07-29T13:36:49.187 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:44.521 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:49.191 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:44.526 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:49.227 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:09:44.561 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:49.244 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:44.528 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:49.244 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:44.578 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:49.246 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:44.531 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:49.249 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:44.583 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:49.277 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:09:44.562 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:36:49.282 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:09:44.617 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:09.331 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:10:04.615 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:09.332 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:10:04.617 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:09.333 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:10:04.667 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:09.338 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:10:04.672 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:09.374 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:10:04.658 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:09.387 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:10:04.721 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:09.410 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:10:04.744 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:09.413 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:10:04.747 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:09.464 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:10:04.798 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:10.088 INFO:tasks.workunit.client.0.smithi040.stdout:Write 2023-07-29T13:37:10.089 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7074 ms) 2023-07-29T13:37:10.089 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2023-07-29T13:37:10.089 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (7182 ms) 2023-07-29T13:37:10.089 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2023-07-29T13:37:10.089 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (45276 ms) 2023-07-29T13:37:10.089 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2023-07-29T13:37:10.089 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7059 ms) 2023-07-29T13:37:10.090 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2023-07-29T13:37:10.090 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (10087 ms) 2023-07-29T13:37:10.090 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2023-07-29T13:37:10.090 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7777 ms) 2023-07-29T13:37:10.090 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2023-07-29T13:37:10.090 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (8065 ms) 2023-07-29T13:37:10.090 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2023-07-29T13:37:10.091 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7967 ms) 2023-07-29T13:37:10.091 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2023-07-29T13:37:10.091 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7052 ms) 2023-07-29T13:37:10.091 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2023-07-29T13:37:10.091 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7612 ms) 2023-07-29T13:37:10.091 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2023-07-29T13:37:10.091 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (80973 ms) 2023-07-29T13:37:10.091 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2023-07-29T13:37:10.092 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6997 ms) 2023-07-29T13:37:10.092 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2023-07-29T13:37:10.092 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7864 ms) 2023-07-29T13:37:10.092 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2023-07-29T13:37:10.092 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (6832 ms) 2023-07-29T13:37:10.092 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2023-07-29T13:37:10.092 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7722 ms) 2023-07-29T13:37:10.092 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2023-07-29T13:37:10.093 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (8095 ms) 2023-07-29T13:37:10.093 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (233635 ms total) 2023-07-29T13:37:10.093 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: 2023-07-29T13:37:10.093 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [----------] Global test environment tear-down 2023-07-29T13:37:10.093 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (476467 ms total) 2023-07-29T13:37:10.093 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio: [ PASSED ] 42 tests. 2023-07-29T13:37:29.525 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:10:24.859 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:29.528 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:10:24.862 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:29.567 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:10:24.901 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:29.577 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:10:24.861 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:29.580 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:10:24.864 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:29.581 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:10:24.915 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:29.582 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:10:24.916 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:29.619 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:10:24.903 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:29.624 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:10:24.959 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:49.640 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:10:44.974 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:49.645 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:10:44.979 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:49.691 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:10:44.976 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:49.693 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:10:44.977 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:49.696 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:10:45.031 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:49.700 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:10:45.034 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:49.701 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:10:45.036 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:49.733 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:10:45.017 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:49.767 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:10:45.102 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:37:57.511 INFO:tasks.workunit.client.0.smithi040.stdout:osAio.RoundTripAppendPP (9067 ms) 2023-07-29T13:37:57.511 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2023-07-29T13:37:57.511 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3423 ms) 2023-07-29T13:37:57.511 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2023-07-29T13:37:57.511 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (35955 ms) 2023-07-29T13:37:57.512 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2023-07-29T13:37:57.512 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3839 ms) 2023-07-29T13:37:57.512 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2023-07-29T13:37:57.512 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: setting pool EIO 2023-07-29T13:37:57.512 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: max_success 16, min_failed 17 2023-07-29T13:37:57.512 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (8695 ms) 2023-07-29T13:37:57.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2023-07-29T13:37:57.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (4531 ms) 2023-07-29T13:37:57.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (319418 ms total) 2023-07-29T13:37:57.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: 2023-07-29T13:37:57.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2023-07-29T13:37:57.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2023-07-29T13:37:57.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3313 ms) 2023-07-29T13:37:57.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2023-07-29T13:37:57.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (11464 ms) 2023-07-29T13:37:57.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2023-07-29T13:37:57.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (17201 ms) 2023-07-29T13:37:57.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2023-07-29T13:37:57.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3408 ms) 2023-07-29T13:37:57.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (35386 ms total) 2023-07-29T13:37:57.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: 2023-07-29T13:37:57.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2023-07-29T13:37:57.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2023-07-29T13:37:57.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3167 ms) 2023-07-29T13:37:57.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3167 ms total) 2023-07-29T13:37:57.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: 2023-07-29T13:37:57.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2023-07-29T13:37:57.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2023-07-29T13:37:57.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (80973 ms) 2023-07-29T13:37:57.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2023-07-29T13:37:57.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (7005 ms) 2023-07-29T13:37:57.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2023-07-29T13:37:57.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (6855 ms) 2023-07-29T13:37:57.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2023-07-29T13:37:57.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7514 ms) 2023-07-29T13:37:57.518 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2023-07-29T13:37:57.518 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (2531 ms) 2023-07-29T13:37:57.518 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2023-07-29T13:37:57.518 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7562 ms) 2023-07-29T13:37:57.518 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2023-07-29T13:37:57.519 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (8274 ms) 2023-07-29T13:37:57.519 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2023-07-29T13:37:57.519 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7332 ms) 2023-07-29T13:37:57.519 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2023-07-29T13:37:57.520 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (8265 ms) 2023-07-29T13:37:57.520 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2023-07-29T13:37:57.520 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (6873 ms) 2023-07-29T13:37:57.520 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2023-07-29T13:37:57.520 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7801 ms) 2023-07-29T13:37:57.521 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2023-07-29T13:37:57.521 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (6982 ms) 2023-07-29T13:37:57.521 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2023-07-29T13:37:57.521 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7947 ms) 2023-07-29T13:38:09.739 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:05.074 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:09.742 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:05.077 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:09.781 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:05.116 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:09.789 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:05.074 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:09.793 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:05.127 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:09.793 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:05.078 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:09.797 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:05.132 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:09.829 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:05.113 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:09.833 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:05.167 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:29.658 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:24.992 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:29.659 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:24.994 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:29.675 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:25.009 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:29.706 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:25.040 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:29.708 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:24.993 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:29.709 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:25.043 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:29.710 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:24.994 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:29.725 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:25.010 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:29.726 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:25.061 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:34.939 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:34.936+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:35.945 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:35.942+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:36.910 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:36.907+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:37.945 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:37.943+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:38.323 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:33.657 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:38.327 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:33.662 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:38.352 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:33.687 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:38.381 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:33.665 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:38.382 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:33.667 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:38.398 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:33.683 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:38.400 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:33.734 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:38.401 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:33.735 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:38.417 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:33.751 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:38.913 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:38.910+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:39.917 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:39.914+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:40.871 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:40.869+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:41.880 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:41.878+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:42.909 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:42.907+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:43.954 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:43.952+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:44.958 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:44.956+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:45.914 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:45.911+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:46.924 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:46.922+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:47.941 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:47.939+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:48.933 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:48.932+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:49.730 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:45.064 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:49.731 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:45.066 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:49.746 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:11:45.081 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:49.780 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:45.114 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:49.785 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:45.119 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:49.798 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:45.083 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:49.800 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:45.084 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:49.804 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:11:45.139 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:49.813 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:11:45.098 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:38:49.935 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:49.933+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:50.904 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:50.902+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:51.883 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:51.881+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:52.893 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:52.891+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:53.886 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:53.884+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:54.911 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:54.909+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:55.901 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:55.900+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:56.903 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:56.901+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:57.896 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:57.894+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:58.868 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:58.866+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:38:59.887 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:38:59.885+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:00.847 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:00.846+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:01.885 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:01.883+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:02.916 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:02.914+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:03.898 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:03.895+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:04.869 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:04.867+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:05.889 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:05.887+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:06.894 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:06.892+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:07.941 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:07.940+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:08.951 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:08.949+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:09.784 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:05.119 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:09.786 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:05.120 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:09.806 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:05.140 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:09.832 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:05.166 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:09.833 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:05.117 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:09.834 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:05.168 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:09.838 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:05.122 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:09.862 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:05.147 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:09.864 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:05.199 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:09.974 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:09.972+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:10.430 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:05.714 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:10.432 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:05.717 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:10.488 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:05.773 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:10.503 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:05.838 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:10.504 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:05.839 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:10.524 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:05.858 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:10.529 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:05.864 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:10.531 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:05.865 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:10.539 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:05.874 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:10.948 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:10.946+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:11.963 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:11.961+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:12.964 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:12.963+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:13.968 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:13.966+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:14.969 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:14.967+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:15.973 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:15.971+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:17.002 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:17.000+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:18.000 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:17.999+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:18.954 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:18.953+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:19.961 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:19.959+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:21.004 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:21.002+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:21.994 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:21.992+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:22.997 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:22.995+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:24.033 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:24.030+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:24.858 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:20.193 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:24.863 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:20.197 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:24.883 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:20.217 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:24.912 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:20.196 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:24.913 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:20.197 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:24.915 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:20.249 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:24.930 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:20.214 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:24.935 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:20.270 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:24.959 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:20.293 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:25.039 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:25.036+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:26.029 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:26.021+0000 12a61640 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:02.761041+0000 front 2023-07-29T13:38:02.759215+0000 (oldest deadline 2023-07-29T13:39:25.046923+0000) 2023-07-29T13:39:26.030 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:26.027+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:26.436 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:39:26.426+0000 12a61640 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:02.513176+0000 front 2023-07-29T13:38:02.509385+0000 (oldest deadline 2023-07-29T13:39:25.400985+0000) 2023-07-29T13:39:27.045 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:27.042+0000 12a61640 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:02.761041+0000 front 2023-07-29T13:38:02.759215+0000 (oldest deadline 2023-07-29T13:39:25.046923+0000) 2023-07-29T13:39:27.045 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:27.043+0000 12a61640 -1 osd.7 398 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14799.0:1 185.2s0 185.d10bd412 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e395) 2023-07-29T13:39:27.090 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:39:27.084+0000 1125e640 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:04.421329+0000 front 2023-07-29T13:38:04.417317+0000 (oldest deadline 2023-07-29T13:39:26.111992+0000) 2023-07-29T13:39:27.320 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:39:27.315+0000 12a61640 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:02.621186+0000 front 2023-07-29T13:38:02.619311+0000 (oldest deadline 2023-07-29T13:39:26.700997+0000) 2023-07-29T13:39:27.405 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:39:27.402+0000 12a61640 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:02.513176+0000 front 2023-07-29T13:38:02.509385+0000 (oldest deadline 2023-07-29T13:39:25.400985+0000) 2023-07-29T13:39:27.734 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:39:27.729+0000 12a61640 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:01.285385+0000 front 2023-07-29T13:38:01.296084+0000 (oldest deadline 2023-07-29T13:39:27.181134+0000) 2023-07-29T13:39:28.055 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:39:28.050+0000 12a61640 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:02.761041+0000 front 2023-07-29T13:38:02.759215+0000 (oldest deadline 2023-07-29T13:39:25.046923+0000) 2023-07-29T13:39:28.123 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:39:28.120+0000 1125e640 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:04.421329+0000 front 2023-07-29T13:38:04.417317+0000 (oldest deadline 2023-07-29T13:39:26.111992+0000) 2023-07-29T13:39:28.227 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:39:28.222+0000 12a61640 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:01.681995+0000 front 2023-07-29T13:38:01.683401+0000 (oldest deadline 2023-07-29T13:39:27.570110+0000) 2023-07-29T13:39:28.319 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:39:28.317+0000 12a61640 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:02.621186+0000 front 2023-07-29T13:38:02.619311+0000 (oldest deadline 2023-07-29T13:39:26.700997+0000) 2023-07-29T13:39:28.434 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:39:28.431+0000 12a61640 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:38:02.513176+0000 front 2023-07-29T13:38:02.509385+0000 (oldest deadline 2023-07-29T13:39:25.400985+0000) 2023-07-29T13:39:34.906 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:30.240 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:34.911 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:30.245 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:34.933 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:30.268 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:34.963 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:30.298 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:34.966 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:30.250 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:34.968 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:30.302 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:34.968 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:30.253 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:34.996 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:30.280 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:35.008 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:30.343 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:37.713 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:39:37.710+0000 14264640 -1 osd.5 406 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:39:40.063 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:35.347 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:40.107 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:35.441 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:40.121 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:35.455 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:40.121 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:35.406 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:40.122 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:35.457 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:40.141 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:35.425 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:40.150 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:35.485 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:40.197 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:35.532 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:40.224 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:35.559 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:55.148 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:50.482 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:55.149 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:50.484 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:55.184 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:50.519 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:55.198 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:50.482 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:55.199 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:50.484 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:55.213 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:50.548 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:55.214 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:50.549 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:55.230 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:50.515 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:55.249 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:50.583 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:56.005 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:51.289 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:56.006 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:51.290 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:56.021 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:12:51.305 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:56.157 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:51.492 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:56.159 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:51.494 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:56.180 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:12:51.515 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:56.230 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:51.564 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:56.231 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:51.566 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:39:56.271 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:12:51.605 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:10.295 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:05.630 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:10.357 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:05.692 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:10.359 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:05.644 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:10.367 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:05.702 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:10.371 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:05.705 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:10.372 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:05.657 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:10.398 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:05.732 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:10.409 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:05.743 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:10.410 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:05.695 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:15.201 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:10.535 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:15.203 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:10.538 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:15.218 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:10.552 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:15.252 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:10.537 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:15.256 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:10.540 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:15.259 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:10.594 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:15.262 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:10.596 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:15.274 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:10.558 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:15.278 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:10.613 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:30.241 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:25.576 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:30.246 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:25.581 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:30.272 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:25.606 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:30.298 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:25.582 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:30.299 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:25.633 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:30.301 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:25.636 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:30.301 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:25.586 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:30.318 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:25.602 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:30.319 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:25.654 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:35.270 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:30.604 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:35.275 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:30.609 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:35.286 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:30.621 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:35.293 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:30.577 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:35.297 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:30.582 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:35.307 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:30.592 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:35.341 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:30.675 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:35.344 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:30.679 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:35.356 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:30.690 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:50.334 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:45.669 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:50.338 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:45.673 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:50.343 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:45.628 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:50.345 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:45.629 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:50.360 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:45.694 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:50.366 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:45.650 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:50.403 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:45.738 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:50.409 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:45.744 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:50.431 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:45.765 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:55.328 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:50.662 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:55.378 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:50.712 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:55.392 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:13:50.726 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:55.425 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:50.760 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:55.427 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:50.712 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:55.429 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:50.713 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:55.440 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:13:50.725 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:55.449 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:50.783 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:40:55.468 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:13:50.802 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:13.176 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:13.174+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:14.225 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:14.223+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:15.178 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:15.176+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:15.414 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:10.749 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:15.417 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:10.752 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:15.440 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:10.775 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:15.467 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:10.801 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:15.468 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:10.753 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:15.469 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:10.804 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:15.474 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:10.759 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:15.490 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:10.824 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:15.496 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:10.780 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:16.170 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:16.168+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:17.134 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:17.133+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:18.163 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:18.162+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:19.204 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:19.202+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:20.246 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:20.244+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:21.228 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:21.225+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:22.226 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:22.224+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:23.206 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:23.204+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:24.179 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:24.178+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:25.202 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:25.200+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:25.444 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:20.779 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:25.446 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:20.781 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:25.463 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:20.798 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:25.496 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:20.830 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:25.500 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:20.834 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:25.500 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:20.784 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:25.515 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:20.800 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:25.519 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:20.853 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:25.530 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:20.814 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:26.209 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:26.208+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:27.207 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:27.206+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:27.971 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:41:27.969+0000 12a61640 -1 osd.7 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:03.197813+0000 front 2023-07-29T13:40:03.194591+0000 (oldest deadline 2023-07-29T13:41:27.891619+0000) 2023-07-29T13:41:28.161 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:28.159+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:28.988 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:41:28.986+0000 12a61640 -1 osd.0 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:02.918358+0000 front 2023-07-29T13:40:02.921078+0000 (oldest deadline 2023-07-29T13:41:28.212468+0000) 2023-07-29T13:41:28.990 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:41:28.988+0000 12a61640 -1 osd.7 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:03.197813+0000 front 2023-07-29T13:40:03.194591+0000 (oldest deadline 2023-07-29T13:41:27.891619+0000) 2023-07-29T13:41:29.144 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:29.142+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:29.970 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:41:29.969+0000 12a61640 -1 osd.7 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:03.197813+0000 front 2023-07-29T13:40:03.194591+0000 (oldest deadline 2023-07-29T13:41:27.891619+0000) 2023-07-29T13:41:29.978 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:41:29.976+0000 12a61640 -1 osd.0 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:02.918358+0000 front 2023-07-29T13:40:02.921078+0000 (oldest deadline 2023-07-29T13:41:28.212468+0000) 2023-07-29T13:41:30.133 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:30.130+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:30.953 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:41:30.951+0000 12a61640 -1 osd.0 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:02.918358+0000 front 2023-07-29T13:40:02.921078+0000 (oldest deadline 2023-07-29T13:41:28.212468+0000) 2023-07-29T13:41:31.021 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:41:31.018+0000 12a61640 -1 osd.7 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:03.197813+0000 front 2023-07-29T13:40:03.194591+0000 (oldest deadline 2023-07-29T13:41:27.891619+0000) 2023-07-29T13:41:31.123 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:31.121+0000 12a61640 -1 osd.6 442 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:31.317 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:41:31.315+0000 1125e640 -1 osd.4 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:05.109401+0000 front 2023-07-29T13:40:05.125190+0000 (oldest deadline 2023-07-29T13:41:30.361409+0000) 2023-07-29T13:41:31.426 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:41:31.423+0000 12a61640 -1 osd.3 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:04.779890+0000 front 2023-07-29T13:40:04.781136+0000 (oldest deadline 2023-07-29T13:41:30.674965+0000) 2023-07-29T13:41:31.921 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:41:31.918+0000 12a61640 -1 osd.0 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:02.918358+0000 front 2023-07-29T13:40:02.921078+0000 (oldest deadline 2023-07-29T13:41:28.212468+0000) 2023-07-29T13:41:31.942 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:41:31.928+0000 12a61640 -1 osd.5 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:05.804698+0000 front 2023-07-29T13:40:05.800906+0000 (oldest deadline 2023-07-29T13:41:31.101070+0000) 2023-07-29T13:41:31.996 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:41:31.994+0000 12a61640 -1 osd.7 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:03.197813+0000 front 2023-07-29T13:40:03.194591+0000 (oldest deadline 2023-07-29T13:41:27.891619+0000) 2023-07-29T13:41:32.152 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:41:32.150+0000 12a61640 -1 osd.2 442 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:40:05.317001+0000 front 2023-07-29T13:40:05.317754+0000 (oldest deadline 2023-07-29T13:41:31.210298+0000) 2023-07-29T13:41:32.294 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:32.283+0000 12a61640 -1 osd.6 443 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:33.241 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:41:33.238+0000 12a61640 -1 osd.6 443 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:2388 194.14 194.14 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e442) 2023-07-29T13:41:33.742 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:41:33.722+0000 12a61640 -1 osd.1 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20275.0:1 193.7s0 193.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e434) 2023-07-29T13:41:34.905 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:41:34.773+0000 12a61640 -1 osd.1 444 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20275.0:1 193.7s0 193.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e434) 2023-07-29T13:41:35.621 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:30.955 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:35.622 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:30.956 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:35.640 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:30.974 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:35.669 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:31.004 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:35.679 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:30.963 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:35.685 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:31.020 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:35.687 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:30.972 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:35.708 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:30.993 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:35.710 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:31.044 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:36.137 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:41:36.135+0000 14264640 -1 osd.1 445 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:41:36.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2023-07-29T13:41:36.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3292 ms) 2023-07-29T13:41:36.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2023-07-29T13:41:36.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (92458 ms) 2023-07-29T13:41:36.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2023-07-29T13:41:36.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (6879 ms) 2023-07-29T13:41:36.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2023-07-29T13:41:36.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7440 ms) 2023-07-29T13:41:36.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2023-07-29T13:41:36.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (6853 ms) 2023-07-29T13:41:36.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2023-07-29T13:41:36.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (8333 ms) 2023-07-29T13:41:36.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2023-07-29T13:41:36.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (93941 ms) 2023-07-29T13:41:36.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (385111 ms total) 2023-07-29T13:41:36.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: 2023-07-29T13:41:36.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [----------] Global test environment tear-down 2023-07-29T13:41:36.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (743082 ms total) 2023-07-29T13:41:36.714 INFO:tasks.workunit.client.0.smithi040.stdout: api_aio_pp: [ PASSED ] 57 tests. 2023-07-29T13:41:40.623 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:35.957 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:40.629 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:35.963 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:40.645 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:35.979 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:40.650 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:35.984 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:40.682 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:35.967 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:40.692 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:35.977 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:40.703 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:36.038 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:40.709 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:35.993 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:40.739 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:36.073 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:55.593 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:50.927 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:55.596 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:50.930 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:55.625 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:50.960 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:55.647 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:50.981 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:55.648 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:50.983 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:55.654 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:50.938 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:55.656 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:50.941 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:55.680 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:51.015 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:55.684 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:50.969 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:56.421 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:51.705 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:56.422 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:51.706 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:56.430 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:14:51.715 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:56.518 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:51.853 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:56.519 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:51.854 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:56.528 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:14:51.863 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:56.571 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:51.905 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:56.572 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:51.906 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:41:56.581 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:14:51.916 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:15.660 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:10.995 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:15.662 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:10.997 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:15.686 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:11.020 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:15.713 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:10.997 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:15.715 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:10.999 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:15.718 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:11.053 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:15.720 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:11.055 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:15.732 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:11.016 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:15.738 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:11.073 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:16.410 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:11.695 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:16.411 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:11.696 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:16.420 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:11.704 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:16.450 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:11.785 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:16.451 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:11.786 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:16.460 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:11.794 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:16.475 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:11.810 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:16.476 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:11.811 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:16.485 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:11.820 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:30.736 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:26.071 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:30.739 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:26.074 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:30.763 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:26.098 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:30.786 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:26.121 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:30.787 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:26.122 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:30.790 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:26.075 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:30.792 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:26.077 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:30.812 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:26.147 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:30.819 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:26.104 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:35.864 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:31.199 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:35.918 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:31.253 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:35.933 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:31.218 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:35.934 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:31.269 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:35.940 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:31.225 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:35.961 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:31.246 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:35.974 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:31.309 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:35.975 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:31.310 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:35.990 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:31.324 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:45.792 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:41.126 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:45.796 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:41.131 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:45.811 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:41.146 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:45.845 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:41.129 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:45.848 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:41.132 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:45.848 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:41.183 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:45.850 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:41.184 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:45.859 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:41.144 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:42:45.864 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:41.199 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:00.835 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:56.170 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:00.837 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:56.172 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:00.859 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:15:56.193 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:00.880 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:56.164 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:00.884 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:56.169 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:00.898 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:56.233 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:00.900 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:56.234 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:00.905 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:15:56.190 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:00.915 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:15:56.249 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:05.847 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:01.181 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:05.873 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:01.186 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:05.874 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:01.198 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:05.896 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:01.230 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:05.896 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:01.181 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:05.898 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:01.183 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:05.900 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:01.235 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:05.910 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:01.194 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:05.913 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:01.247 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:21.012 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:16.347 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:21.016 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:16.350 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:21.036 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:16.371 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:21.063 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:16.347 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:21.064 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:16.349 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:21.072 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:16.406 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:21.074 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:16.408 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:21.078 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:16.362 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:21.088 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:16.423 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:25.917 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:21.252 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:25.920 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:21.255 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:25.930 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:21.264 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:25.969 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:21.254 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:25.970 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:21.305 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:25.971 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:21.256 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:25.971 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:21.306 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:25.980 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:21.265 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:25.981 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:21.316 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:40.954 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:36.289 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:40.957 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:36.292 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:40.980 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:36.315 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:41.006 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:36.290 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:41.007 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:36.292 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:41.015 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:36.349 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:41.017 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:36.352 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:41.021 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:36.306 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:41.033 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:36.367 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:42.449 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:37.783 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:42.452 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:37.786 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:42.460 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:37.745 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:42.462 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:37.796 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:42.502 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:37.787 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:42.507 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:37.842 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:42.513 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:37.848 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:42.518 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:37.803 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:42.525 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:37.860 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:45.971 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:41.305 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:45.974 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:41.309 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:45.985 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:41.319 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:46.020 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:41.305 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:46.022 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:41.307 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:46.024 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:41.358 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:46.025 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:41.360 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:46.034 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:41.318 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:43:46.035 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:41.370 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:01.122 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:56.457 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:01.125 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:56.460 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:01.167 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:16:56.501 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:01.181 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:56.465 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:01.183 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:56.467 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:01.185 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:56.520 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:01.187 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:56.522 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:01.210 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:16:56.494 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:01.224 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:16:56.559 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:16.184 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:11.519 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:16.186 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:11.521 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:16.215 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:11.549 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:16.235 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:11.520 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:16.236 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:11.570 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:16.236 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:11.521 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:16.238 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:11.573 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:16.264 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:11.549 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:16.270 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:11.604 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:18.177 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2023-07-29T13:44:18.178 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [----------] Global test environment set-up. 2023-07-29T13:44:18.178 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2023-07-29T13:44:18.178 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: seed 90886 2023-07-29T13:44:18.178 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2023-07-29T13:44:18.178 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (27609 ms) 2023-07-29T13:44:18.178 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2023-07-29T13:44:18.179 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (39267 ms) 2023-07-29T13:44:18.179 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2023-07-29T13:44:18.179 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (887 ms) 2023-07-29T13:44:18.179 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (67763 ms total) 2023-07-29T13:44:18.179 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: 2023-07-29T13:44:18.179 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2023-07-29T13:44:18.179 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2023-07-29T13:44:18.179 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (49238 ms) 2023-07-29T13:44:18.180 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2023-07-29T13:44:18.180 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (53855 ms) 2023-07-29T13:44:18.180 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2023-07-29T13:44:18.180 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (20026 ms) 2023-07-29T13:44:18.180 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2023-07-29T13:44:18.180 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: my_snaps [4] 2023-07-29T13:44:18.180 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: my_snaps [5,4] 2023-07-29T13:44:18.181 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: my_snaps [6,5,4] 2023-07-29T13:44:18.181 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: my_snaps [7,6,5,4] 2023-07-29T13:44:18.181 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: promoting some heads 2023-07-29T13:44:18.181 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: promoting from clones for snap 7 2023-07-29T13:44:18.181 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: promoting from clones for snap 6 2023-07-29T13:44:18.181 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: promoting from clones for snap 5 2023-07-29T13:44:18.181 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: promoting from clones for snap 4 2023-07-29T13:44:18.181 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: waiting for scrubs... 2023-07-29T13:44:18.182 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: done waiting 2023-07-29T13:44:18.182 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (293025 ms) 2023-07-29T13:44:18.182 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2023-07-29T13:44:18.182 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (11213 ms) 2023-07-29T13:44:18.182 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2023-07-29T13:44:18.182 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8737 ms) 2023-07-29T13:44:18.182 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2023-07-29T13:44:18.182 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8577 ms) 2023-07-29T13:44:18.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2023-07-29T13:44:18.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (97107 ms) 2023-07-29T13:44:18.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2023-07-29T13:44:18.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (12261 ms) 2023-07-29T13:44:18.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2023-07-29T13:44:18.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (9786 ms) 2023-07-29T13:44:18.183 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2023-07-29T13:44:18.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (11662 ms) 2023-07-29T13:44:18.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2023-07-29T13:44:18.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (94835 ms) 2023-07-29T13:44:18.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2023-07-29T13:44:18.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (9731 ms) 2023-07-29T13:44:18.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2023-07-29T13:44:18.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (38206 ms) 2023-07-29T13:44:18.184 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2023-07-29T13:44:18.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (89507 ms) 2023-07-29T13:44:18.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2023-07-29T13:44:18.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (8849 ms) 2023-07-29T13:44:18.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2023-07-29T13:44:18.185 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (12101 ms) 2023-07-29T13:44:21.204 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:16.538 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:21.208 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:16.543 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:21.221 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:16.556 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:21.253 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:16.537 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:21.254 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:16.538 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:21.266 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:16.551 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:21.278 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:16.612 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:21.280 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:16.614 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:21.293 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:16.627 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:31.174 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:26.508 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:31.174 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:26.459 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:31.175 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:26.460 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:31.178 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:26.513 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:31.184 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:26.518 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:31.207 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:26.492 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:31.218 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:26.553 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:31.249 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:26.584 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:31.274 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:26.608 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:40.698 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2023-07-29T13:44:40.698 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: hmm, no HitSet yet 2023-07-29T13:44:40.698 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: ok, hit_set contains 200:602f83fe:::foo:head 2023-07-29T13:44:40.698 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9564 ms) 2023-07-29T13:44:40.698 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2023-07-29T13:44:40.699 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg_num = 32 2023-07-29T13:44:40.699 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 0 ls 1690638275,0 2023-07-29T13:44:40.699 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 1 ls 1690638276,0 2023-07-29T13:44:40.699 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 2 ls 1690638275,0 2023-07-29T13:44:40.699 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 3 ls 1690638275,0 2023-07-29T13:44:40.699 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 4 ls 1690638276,0 2023-07-29T13:44:40.699 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 5 ls 1690638275,0 2023-07-29T13:44:40.700 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 6 ls 1690638275,0 2023-07-29T13:44:40.700 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 7 ls 1690638275,0 2023-07-29T13:44:40.700 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 8 ls 1690638275,0 2023-07-29T13:44:40.700 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 9 ls 1690638275,0 2023-07-29T13:44:40.700 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 10 ls 1690638275,0 2023-07-29T13:44:40.700 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 11 ls 1690638276,0 2023-07-29T13:44:40.700 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 12 ls 1690638275,0 2023-07-29T13:44:40.700 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 13 ls 1690638275,0 2023-07-29T13:44:40.701 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 14 ls 1690638275,0 2023-07-29T13:44:40.701 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 15 ls 1690638275,0 2023-07-29T13:44:40.701 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 16 ls 1690638275,0 2023-07-29T13:44:40.701 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 17 ls 1690638275,0 2023-07-29T13:44:40.701 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 18 ls 1690638275,0 2023-07-29T13:44:40.701 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 19 ls 1690638275,0 2023-07-29T13:44:40.701 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 20 ls 1690638275,0 2023-07-29T13:44:40.702 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 21 ls 1690638275,0 2023-07-29T13:44:40.702 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 22 ls 1690638275,0 2023-07-29T13:44:40.702 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 23 ls 1690638275,0 2023-07-29T13:44:40.702 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 24 ls 1690638275,0 2023-07-29T13:44:40.702 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 25 ls 1690638275,0 2023-07-29T13:44:40.702 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 26 ls 1690638275,0 2023-07-29T13:44:40.702 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 27 ls 1690638275,0 2023-07-29T13:44:40.702 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 28 ls 1690638275,0 2023-07-29T13:44:40.703 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 29 ls 1690638275,0 2023-07-29T13:44:40.703 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 30 ls 1690638275,0 2023-07-29T13:44:40.703 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg 31 ls 1690638275,0 2023-07-29T13:44:40.703 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: pg_num = 32 2023-07-29T13:44:40.703 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:6cac518f:::0:head 2023-07-29T13:44:40.703 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:02547ec2:::1:head 2023-07-29T13:44:40.703 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:f905c69b:::2:head 2023-07-29T13:44:40.703 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:cfc208b3:::3:head 2023-07-29T13:44:40.704 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:d83876eb:::4:head 2023-07-29T13:44:40.704 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:b29083e3:::5:head 2023-07-29T13:44:40.704 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:c4fdafeb:::6:head 2023-07-29T13:44:40.704 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:5c6b0b28:::7:head 2023-07-29T13:44:40.704 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:bd63b0f1:::8:head 2023-07-29T13:44:40.704 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:e960b815:::9:head 2023-07-29T13:44:40.704 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:52ea6a34:::10:head 2023-07-29T13:44:40.705 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:89d3ae78:::11:head 2023-07-29T13:44:40.705 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:de5d7c5f:::12:head 2023-07-29T13:44:40.705 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:566253c9:::13:head 2023-07-29T13:44:40.705 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:62a1935d:::14:head 2023-07-29T13:44:40.705 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:863748b0:::15:head 2023-07-29T13:44:40.705 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:3958e169:::16:head 2023-07-29T13:44:40.705 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:4d4dabf9:::17:head 2023-07-29T13:44:40.705 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:8391935d:::18:head 2023-07-29T13:44:40.706 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:28883081:::19:head 2023-07-29T13:44:40.706 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:69259c59:::20:head 2023-07-29T13:44:40.706 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:4bdb80b7:::21:head 2023-07-29T13:44:40.706 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:a11c5d71:::22:head 2023-07-29T13:44:40.706 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:271af37b:::23:head 2023-07-29T13:44:40.706 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:95b121be:::24:head 2023-07-29T13:44:40.706 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:58d1031b:::25:head 2023-07-29T13:44:40.706 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:0a050783:::26:head 2023-07-29T13:44:40.707 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:c709704c:::27:head 2023-07-29T13:44:40.707 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:cbe56eaf:::28:head 2023-07-29T13:44:40.707 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:86b4b162:::29:head 2023-07-29T13:44:40.707 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:70d89383:::30:head 2023-07-29T13:44:40.707 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:dd450c7c:::31:head 2023-07-29T13:44:40.707 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:6d5729b1:::32:head 2023-07-29T13:44:40.707 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:c388f3fb:::33:head 2023-07-29T13:44:40.707 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:56cfea31:::34:head 2023-07-29T13:44:40.708 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:9dbc1bf7:::35:head 2023-07-29T13:44:40.708 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:40b74ccd:::36:head 2023-07-29T13:44:40.708 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:4d5aaf42:::37:head 2023-07-29T13:44:40.708 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:920f362c:::38:head 2023-07-29T13:44:40.708 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:6cc53222:::39:head 2023-07-29T13:44:40.708 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:9cad833f:::40:head 2023-07-29T13:44:40.708 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:1ea84d41:::41:head 2023-07-29T13:44:40.709 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:c4480ef6:::42:head 2023-07-29T13:44:40.709 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:a694361e:::43:head 2023-07-29T13:44:40.709 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:d1bd33e9:::44:head 2023-07-29T13:44:40.709 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:ddc2cd5d:::45:head 2023-07-29T13:44:40.709 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:2b782207:::46:head 2023-07-29T13:44:40.709 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:7b187fca:::47:head 2023-07-29T13:44:40.709 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:90ecdf6f:::48:head 2023-07-29T13:44:40.709 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:a5ed95fe:::49:head 2023-07-29T13:44:40.710 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:ea0eaa55:::50:head 2023-07-29T13:44:40.710 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:f33ef17b:::51:head 2023-07-29T13:44:40.710 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:a0d1b2f6:::52:head 2023-07-29T13:44:40.710 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:60c5229e:::53:head 2023-07-29T13:44:40.710 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:edcbc575:::54:head 2023-07-29T13:44:40.710 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:102cf253:::55:head 2023-07-29T13:44:40.710 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:efb7fb0b:::56:head 2023-07-29T13:44:40.710 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:50d0a326:::57:head 2023-07-29T13:44:40.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:d4dc5daf:::58:head 2023-07-29T13:44:40.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:3a130462:::59:head 2023-07-29T13:44:40.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:ec87ed71:::60:head 2023-07-29T13:44:40.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:d5bc9454:::61:head 2023-07-29T13:44:40.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:3ddfe313:::62:head 2023-07-29T13:44:40.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:7c2816b9:::63:head 2023-07-29T13:44:40.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:47e00e4d:::64:head 2023-07-29T13:44:40.711 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:c6410c18:::65:head 2023-07-29T13:44:40.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:b48ed237:::66:head 2023-07-29T13:44:40.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:cd63ad31:::67:head 2023-07-29T13:44:40.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:b179e92b:::68:head 2023-07-29T13:44:40.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:0d9f741a:::69:head 2023-07-29T13:44:40.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:6d3352ae:::70:head 2023-07-29T13:44:40.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:c6d5c19e:::71:head 2023-07-29T13:44:40.712 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:bc4729c3:::72:head 2023-07-29T13:44:40.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:77e930b9:::73:head 2023-07-29T13:44:40.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:0abeecfd:::74:head 2023-07-29T13:44:40.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:b7c37e15:::75:head 2023-07-29T13:44:40.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:b6378398:::76:head 2023-07-29T13:44:40.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:02bd68de:::77:head 2023-07-29T13:44:40.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:cc795d2d:::78:head 2023-07-29T13:44:40.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:630d4fea:::79:head 2023-07-29T13:44:40.713 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:e0d29ef5:::80:head 2023-07-29T13:44:40.714 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:fd6f13d2:::81:head 2023-07-29T13:44:40.714 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:606461d5:::82:head 2023-07-29T13:44:40.714 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:eadbdc43:::83:head 2023-07-29T13:44:40.714 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:8761d0bb:::84:head 2023-07-29T13:44:40.714 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:9ef0186f:::85:head 2023-07-29T13:44:40.714 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:e0d41294:::86:head 2023-07-29T13:44:40.714 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:961de695:::87:head 2023-07-29T13:44:40.715 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:1423148f:::88:head 2023-07-29T13:44:40.715 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:633a8fa2:::89:head 2023-07-29T13:44:40.715 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:a8653809:::90:head 2023-07-29T13:44:40.715 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:3dac8b33:::91:head 2023-07-29T13:44:40.715 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:35aad435:::92:head 2023-07-29T13:44:40.715 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:f6dcc343:::93:head 2023-07-29T13:44:40.715 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:dbbdad87:::94:head 2023-07-29T13:44:40.715 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:1cb48ce0:::95:head 2023-07-29T13:44:40.716 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:03cd461c:::96:head 2023-07-29T13:44:40.716 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:17a4ea99:::97:head 2023-07-29T13:44:40.716 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:9993c9a7:::98:head 2023-07-29T13:44:40.716 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:6394211c:::99:head 2023-07-29T13:44:40.716 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:94c7ae57:::100:head 2023-07-29T13:44:40.716 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:6fdee5bb:::101:head 2023-07-29T13:44:40.716 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:9a477fd1:::102:head 2023-07-29T13:44:40.717 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:eb850916:::103:head 2023-07-29T13:44:40.717 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:affc56b9:::104:head 2023-07-29T13:44:40.717 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:b42dc814:::105:head 2023-07-29T13:44:40.717 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:f319f8f0:::106:head 2023-07-29T13:44:40.717 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:9a40b9de:::107:head 2023-07-29T13:44:40.717 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:8b524f28:::108:head 2023-07-29T13:44:40.717 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:e3de589f:::109:head 2023-07-29T13:44:40.717 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:90f90a5b:::110:head 2023-07-29T13:44:40.718 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:a7b4f1d7:::111:head 2023-07-29T13:44:40.718 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:af51766e:::112:head 2023-07-29T13:44:40.718 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:b6f90bd1:::113:head 2023-07-29T13:44:40.718 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:e0261208:::114:head 2023-07-29T13:44:40.718 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:c9569ef7:::115:head 2023-07-29T13:44:40.718 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:61bebe50:::116:head 2023-07-29T13:44:40.718 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:fe93412b:::117:head 2023-07-29T13:44:40.718 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:d3d38bee:::118:head 2023-07-29T13:44:40.719 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:3100ba0c:::119:head 2023-07-29T13:44:40.719 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:d0560ada:::120:head 2023-07-29T13:44:40.719 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:f0ea8b35:::121:head 2023-07-29T13:44:40.719 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:766f231a:::122:head 2023-07-29T13:44:40.719 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:a07a2582:::123:head 2023-07-29T13:44:40.719 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:bd7c6b3a:::124:head 2023-07-29T13:44:40.719 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:fb2ddaff:::125:head 2023-07-29T13:44:40.720 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:4408e1fe:::126:head 2023-07-29T13:44:40.720 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:ee1df7a7:::127:head 2023-07-29T13:44:40.720 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:c3002909:::128:head 2023-07-29T13:44:40.720 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:4f48ffa9:::129:head 2023-07-29T13:44:40.720 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:edf38733:::130:head 2023-07-29T13:44:40.720 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:c08425c0:::131:head 2023-07-29T13:44:40.720 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:5f902d98:::132:head 2023-07-29T13:44:40.720 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:41ea2c93:::133:head 2023-07-29T13:44:40.721 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:813cee13:::134:head 2023-07-29T13:44:40.721 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:0131818d:::135:head 2023-07-29T13:44:40.721 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:26ba5a85:::136:head 2023-07-29T13:44:40.721 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:381b8a5a:::137:head 2023-07-29T13:44:40.721 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:28797e47:::138:head 2023-07-29T13:44:40.721 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:bfca7f22:::139:head 2023-07-29T13:44:40.721 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:36807075:::140:head 2023-07-29T13:44:40.721 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:80b03975:::141:head 2023-07-29T13:44:40.722 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:5c15709b:::142:head 2023-07-29T13:44:40.722 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:f39ea15e:::143:head 2023-07-29T13:44:40.722 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:ea992956:::144:head 2023-07-29T13:44:40.722 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:48887b1c:::145:head 2023-07-29T13:44:40.722 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:9f24a9dd:::146:head 2023-07-29T13:44:40.722 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:987f100b:::147:head 2023-07-29T13:44:40.722 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:d2dd3581:::148:head 2023-07-29T13:44:40.722 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:7fed1808:::149:head 2023-07-29T13:44:40.723 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:c80b70e9:::150:head 2023-07-29T13:44:40.723 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:85ed90f9:::151:head 2023-07-29T13:44:40.723 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:36428b24:::152:head 2023-07-29T13:44:40.723 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:d044c34a:::153:head 2023-07-29T13:44:40.723 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:7c18bf58:::154:head 2023-07-29T13:44:40.723 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:d1c21232:::155:head 2023-07-29T13:44:40.723 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:a7a3c575:::156:head 2023-07-29T13:44:40.723 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:87da0633:::157:head 2023-07-29T13:44:40.724 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:d5ac3822:::158:head 2023-07-29T13:44:40.724 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:3f20522d:::159:head 2023-07-29T13:44:40.724 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:6ca26563:::160:head 2023-07-29T13:44:40.724 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:532ce135:::161:head 2023-07-29T13:44:40.724 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:c78863e6:::162:head 2023-07-29T13:44:41.129 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:36.464 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:41.178 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:36.463 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:41.180 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:36.464 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:41.183 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:36.518 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:41.204 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:36.489 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:41.206 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:36.540 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:41.238 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:36.573 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:41.240 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:36.575 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:41.260 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:36.594 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:46.288 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:41.622 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:46.353 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:41.687 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:46.365 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:41.699 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:46.401 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:41.736 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:46.402 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:41.737 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:46.403 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:41.688 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:46.405 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:41.689 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:46.414 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:41.749 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:44:46.417 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:41.702 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.273 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:56.608 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.275 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:56.610 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.305 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:56.640 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.324 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:56.609 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.327 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:56.662 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.328 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:56.613 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.330 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:56.664 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.354 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:56.639 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.355 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:56.690 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.964 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:57.298 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.965 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:57.300 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.973 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:17:57.308 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.975 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:57.310 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.976 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:57.260 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.976 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:57.311 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.977 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:57.262 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.984 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:17:57.319 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:01.985 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:17:57.269 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:21.474 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:16.809 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:21.479 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:16.814 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:21.513 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:16.848 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:21.534 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:16.819 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:21.535 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:16.820 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:21.540 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:16.874 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:21.541 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:16.876 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:21.562 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:16.847 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:21.576 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:16.911 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:36.335 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:31.669 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:36.385 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:31.669 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:36.386 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:31.671 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:36.391 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:31.725 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:36.419 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:31.704 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:36.429 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:31.764 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:36.440 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:31.775 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:36.445 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:31.780 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:36.477 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:31.812 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:41.421 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:36.756 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:41.423 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:36.758 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:41.435 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:36.770 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:41.470 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:36.755 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:41.472 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:36.807 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:41.472 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:36.757 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:41.474 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:36.809 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:41.483 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:36.768 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:41.488 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:36.823 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:52.582 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:7462ddf6:::163:head 2023-07-29T13:45:52.582 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:5d165639:::164:head 2023-07-29T13:45:52.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:f43765fc:::165:head 2023-07-29T13:45:52.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:b4c720e9:::166:head 2023-07-29T13:45:52.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:e694b040:::167:head 2023-07-29T13:45:52.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:afa38db2:::168:head 2023-07-29T13:45:52.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:77ba9f53:::169:head 2023-07-29T13:45:52.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:87495034:::170:head 2023-07-29T13:45:52.583 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:7c96bf0e:::171:head 2023-07-29T13:45:52.584 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:dbe346cc:::172:head 2023-07-29T13:45:52.584 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:e943ec24:::173:head 2023-07-29T13:45:52.584 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:f97a9c0c:::174:head 2023-07-29T13:45:52.584 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:6f26e74d:::175:head 2023-07-29T13:45:52.584 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:4f95e106:::176:head 2023-07-29T13:45:52.584 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:0e6f2f8f:::177:head 2023-07-29T13:45:52.584 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:05db05f1:::178:head 2023-07-29T13:45:52.585 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:38a78d66:::179:head 2023-07-29T13:45:52.585 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:d095610b:::180:head 2023-07-29T13:45:52.585 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:a1a9d709:::181:head 2023-07-29T13:45:52.585 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:1e5d39db:::182:head 2023-07-29T13:45:52.585 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:f7df4fb9:::183:head 2023-07-29T13:45:52.585 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:03a7f161:::184:head 2023-07-29T13:45:52.585 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:ba70721e:::185:head 2023-07-29T13:45:52.585 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:28e5662d:::186:head 2023-07-29T13:45:52.586 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:973d52de:::187:head 2023-07-29T13:45:52.586 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:4303eb1c:::188:head 2023-07-29T13:45:52.586 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:b990b48e:::189:head 2023-07-29T13:45:52.586 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:29b8165b:::190:head 2023-07-29T13:45:52.586 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:3547f197:::191:head 2023-07-29T13:45:52.586 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:7e260936:::192:head 2023-07-29T13:45:52.586 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:1abec7b1:::193:head 2023-07-29T13:45:52.586 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:10fdda93:::194:head 2023-07-29T13:45:52.587 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:15817eea:::195:head 2023-07-29T13:45:52.587 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:770bab57:::196:head 2023-07-29T13:45:52.587 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:ed9e13e7:::197:head 2023-07-29T13:45:52.587 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:71471a8f:::198:head 2023-07-29T13:45:52.587 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: checking for 201:10fb1d02:::199:head 2023-07-29T13:45:52.587 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (22258 ms) 2023-07-29T13:45:52.587 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2023-07-29T13:45:52.587 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,0 2023-07-29T13:45:52.588 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: first is 1690638298 2023-07-29T13:45:52.588 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,0 2023-07-29T13:45:52.588 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,0 2023-07-29T13:45:52.588 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,1690638301,1690638302,0 2023-07-29T13:45:52.588 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,1690638301,1690638302,0 2023-07-29T13:45:52.588 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,1690638301,1690638302,0 2023-07-29T13:45:52.588 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,1690638301,1690638302,1690638304,1690638305,0 2023-07-29T13:45:52.589 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,1690638301,1690638302,1690638304,1690638305,0 2023-07-29T13:45:52.589 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,1690638301,1690638302,1690638304,1690638305,0 2023-07-29T13:45:52.589 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,1690638301,1690638302,1690638304,1690638305,1690638307,1690638308,0 2023-07-29T13:45:52.589 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,1690638301,1690638302,1690638304,1690638305,1690638307,1690638308,0 2023-07-29T13:45:52.589 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638298,1690638301,1690638302,1690638304,1690638305,1690638307,1690638308,0 2023-07-29T13:45:52.589 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690638302,1690638304,1690638305,1690638307,1690638308,1690638310,1690638311,0 2023-07-29T13:45:52.589 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: first now 1690638302, trimmed 2023-07-29T13:45:52.589 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (23398 ms) 2023-07-29T13:45:52.590 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2023-07-29T13:45:52.590 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: foo0 2023-07-29T13:45:52.590 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: verifying foo0 is eventually promoted 2023-07-29T13:45:52.590 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (16760 ms) 2023-07-29T13:45:52.590 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2023-07-29T13:45:52.742 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:48.076 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:52.747 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:48.082 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:52.768 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:48.103 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:52.801 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:48.136 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:52.802 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:48.137 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:52.810 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:48.094 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:52.811 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:48.096 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:52.823 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:48.158 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:52.833 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:48.117 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:56.450 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:51.785 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:56.453 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:51.788 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:56.471 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:51.806 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:56.501 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:51.786 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:56.503 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:51.787 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:56.507 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:51.842 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:56.511 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:51.845 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:56.514 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:51.799 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:45:56.525 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:51.859 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:01.461 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:56.795 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:01.463 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:56.798 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:01.475 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:18:56.810 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:01.509 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:56.794 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:01.510 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:56.795 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:01.512 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:56.847 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:01.513 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:56.848 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:01.520 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:18:56.805 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:01.525 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:18:56.860 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:16.513 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:11.847 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:16.514 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:11.849 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:16.529 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:11.864 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:16.561 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:11.895 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:16.562 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:11.847 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:16.563 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:11.848 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:16.564 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:11.899 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:16.578 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:11.863 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:16.583 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:11.918 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:21.522 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:16.857 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:21.523 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:16.858 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:21.533 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:16.868 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:21.569 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:16.854 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:21.571 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:16.906 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:21.571 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:16.856 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:21.572 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:16.907 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:21.582 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:16.867 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:21.583 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:16.918 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:36.563 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:31.898 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:36.565 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:31.899 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:36.579 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:31.914 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:36.610 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:31.945 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:36.613 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:31.898 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:36.613 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:31.948 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:36.615 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:31.900 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:36.629 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:31.964 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:36.631 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:31.916 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:41.640 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:36.975 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:41.642 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:36.977 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:41.652 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:36.987 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:41.684 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:36.969 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:41.686 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:37.021 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:41.688 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:36.972 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:41.691 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:37.026 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:41.699 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:36.984 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:41.703 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:37.038 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:56.637 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:51.972 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:56.640 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:51.975 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:56.653 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:19:51.988 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:56.686 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:51.970 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:56.688 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:51.973 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:56.689 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:52.024 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:56.691 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:52.026 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:56.702 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:19:51.987 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:46:56.709 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:19:52.044 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:06.684 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:02.019 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:06.686 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:02.021 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:06.705 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:02.040 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:06.730 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:02.014 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:06.732 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:02.017 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:06.738 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:02.072 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:06.741 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:02.076 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:06.744 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:02.029 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:06.754 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:02.089 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:15.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:47:15.187+0000 12a61640 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:50.330184+0000 front 2023-07-29T13:45:50.320595+0000 (oldest deadline 2023-07-29T13:47:14.998549+0000) 2023-07-29T13:47:15.984 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:47:15.982+0000 12a61640 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:54.211547+0000 front 2023-07-29T13:45:54.207002+0000 (oldest deadline 2023-07-29T13:47:15.300463+0000) 2023-07-29T13:47:16.167 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:47:16.166+0000 12a61640 -1 osd.3 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:52.530648+0000 front 2023-07-29T13:45:52.527663+0000 (oldest deadline 2023-07-29T13:47:16.021310+0000) 2023-07-29T13:47:16.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:47:16.213+0000 12a61640 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:50.330184+0000 front 2023-07-29T13:45:50.320595+0000 (oldest deadline 2023-07-29T13:47:14.998549+0000) 2023-07-29T13:47:16.709 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:12.043 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:16.710 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:12.045 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:16.722 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:12.057 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:16.755 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:47:16.751+0000 12a61640 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:51.685457+0000 front 2023-07-29T13:45:51.680991+0000 (oldest deadline 2023-07-29T13:47:16.374444+0000) 2023-07-29T13:47:16.759 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:12.044 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:16.760 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:12.045 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:16.761 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:12.096 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:16.763 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:12.098 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:16.771 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:12.056 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:16.774 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:12.109 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:16.800 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:47:16.798+0000 1125e640 -1 osd.4 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:54.317995+0000 front 2023-07-29T13:45:54.315936+0000 (oldest deadline 2023-07-29T13:47:16.603747+0000) 2023-07-29T13:47:17.034 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:47:17.032+0000 12a61640 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:54.211547+0000 front 2023-07-29T13:45:54.207002+0000 (oldest deadline 2023-07-29T13:47:15.300463+0000) 2023-07-29T13:47:17.126 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:47:17.125+0000 12a61640 -1 osd.3 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:52.530648+0000 front 2023-07-29T13:45:52.527663+0000 (oldest deadline 2023-07-29T13:47:16.021310+0000) 2023-07-29T13:47:17.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:47:17.223+0000 12a61640 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:50.330184+0000 front 2023-07-29T13:45:50.320595+0000 (oldest deadline 2023-07-29T13:47:14.998549+0000) 2023-07-29T13:47:17.797 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:47:17.794+0000 12a61640 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:51.685457+0000 front 2023-07-29T13:45:51.680991+0000 (oldest deadline 2023-07-29T13:47:16.374444+0000) 2023-07-29T13:47:17.829 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:47:17.826+0000 1125e640 -1 osd.4 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:54.317995+0000 front 2023-07-29T13:45:54.315936+0000 (oldest deadline 2023-07-29T13:47:16.603747+0000) 2023-07-29T13:47:17.835 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:47:17.832+0000 12a61640 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:53.510555+0000 front 2023-07-29T13:45:53.509354+0000 (oldest deadline 2023-07-29T13:47:17.592996+0000) 2023-07-29T13:47:18.091 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:47:18.090+0000 12a61640 -1 osd.3 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:52.530648+0000 front 2023-07-29T13:45:52.527663+0000 (oldest deadline 2023-07-29T13:47:16.021310+0000) 2023-07-29T13:47:18.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:47:18.248+0000 12a61640 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:45:50.330184+0000 front 2023-07-29T13:45:50.320595+0000 (oldest deadline 2023-07-29T13:47:14.998549+0000) 2023-07-29T13:47:26.909 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:22.243 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:26.910 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:22.245 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:26.930 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:22.215 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:26.968 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:22.303 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:26.975 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:22.260 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:26.978 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:22.313 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:27.012 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:22.346 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:27.043 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:22.378 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:27.046 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:22.331 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:31.983 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:27.318 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:31.985 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:27.320 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:31.997 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:27.332 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:32.031 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:27.316 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:32.032 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:27.317 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:32.033 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:27.368 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:32.034 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:27.370 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:32.045 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:27.330 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:32.047 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:27.382 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:42.680 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:47:42.676+0000 12a61640 -1 osd.5 549 heartbeat_check: no reply from 172.21.15.40:6820 osd.1 since back 2023-07-29T13:45:53.783409+0000 front 2023-07-29T13:45:53.779180+0000 (oldest deadline 2023-07-29T13:47:14.920590+0000) 2023-07-29T13:47:42.691 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:47:42.676+0000 12a61640 -1 osd.5 549 heartbeat_check: no reply from 172.21.15.40:6804 osd.2 since back 2023-07-29T13:45:53.769859+0000 front 2023-07-29T13:45:53.777694+0000 (oldest deadline 2023-07-29T13:47:14.920590+0000) 2023-07-29T13:47:42.691 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:47:42.676+0000 12a61640 -1 osd.5 549 heartbeat_check: no reply from 172.21.15.40:6828 osd.3 since back 2023-07-29T13:45:53.776805+0000 front 2023-07-29T13:45:54.944708+0000 (oldest deadline 2023-07-29T13:47:14.920590+0000) 2023-07-29T13:47:42.691 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:47:42.677+0000 12a61640 -1 osd.5 549 heartbeat_check: no reply from 172.21.15.45:6830 osd.4 since back 2023-07-29T13:45:53.774011+0000 front 2023-07-29T13:45:53.771508+0000 (oldest deadline 2023-07-29T13:47:14.920590+0000) 2023-07-29T13:47:42.708 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:47:42.677+0000 12a61640 -1 osd.5 549 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T13:45:53.764884+0000 front 2023-07-29T13:45:53.763086+0000 (oldest deadline 2023-07-29T13:47:14.920590+0000) 2023-07-29T13:47:42.713 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:47:42.677+0000 12a61640 -1 osd.5 549 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:45:54.947624+0000 front 2023-07-29T13:45:53.766442+0000 (oldest deadline 2023-07-29T13:47:14.920590+0000) 2023-07-29T13:47:45.785 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:47:45.783+0000 14264640 -1 osd.5 567 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:47:47.071 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:42.406 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:47.077 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:42.412 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:47.115 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:42.450 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:47.132 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:42.417 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:47.133 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:42.418 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:47.149 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:42.484 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:47.159 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:42.494 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:47.160 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:42.445 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:47:47.197 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:42.532 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:01.879 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:57.214 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:01.933 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:57.218 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:01.934 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:57.219 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:01.940 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:57.275 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:01.957 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:20:57.242 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:01.977 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:20:57.312 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:01.997 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:57.332 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:01.999 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:57.334 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:02.038 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:20:57.372 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:06.960 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:02.295 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:06.965 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:02.300 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:06.982 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:02.316 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:07.017 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:02.301 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:07.021 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:02.306 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:07.023 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:02.358 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:07.024 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:02.360 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:07.034 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:02.319 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:07.038 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:02.373 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:17.012 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:12.347 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:17.015 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:12.350 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:17.037 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:12.371 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:17.066 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:12.350 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:17.068 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:12.353 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:17.069 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:12.404 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:17.070 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:12.405 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:17.088 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:12.373 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:17.091 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:12.426 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:27.158 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:22.493 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:27.161 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:22.496 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:27.176 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:22.511 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:27.212 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:22.497 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:27.212 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:22.547 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:27.213 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:22.548 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:27.217 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:22.502 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:27.226 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:22.561 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:27.230 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:22.515 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:31.843 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:27.178 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:31.853 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:27.188 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:31.863 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:27.148 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:31.864 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:27.199 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:31.866 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:27.201 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:31.908 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:27.193 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:31.926 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:27.211 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:31.952 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:27.287 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:31.959 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:27.294 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:37.168 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:32.503 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:37.171 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:32.506 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:37.186 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:32.521 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:37.218 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:32.503 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:37.219 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:32.504 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:37.223 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:32.558 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:37.227 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:32.562 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:37.235 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:32.520 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:37.242 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:32.577 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:47.228 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:42.563 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:47.232 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:42.567 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:47.250 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:42.585 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:47.284 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:42.569 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:47.287 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:42.572 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:47.297 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:42.632 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:47.299 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:42.634 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:47.305 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:42.590 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:47.319 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:42.654 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:52.344 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:47.678 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:52.396 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:47.731 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:52.409 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:21:47.744 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:52.451 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:47.736 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:52.454 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:47.739 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:52.463 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:47.798 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:52.465 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:47.800 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:52.469 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:21:47.754 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:48:52.477 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:21:47.812 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:07.312 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:02.647 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:07.314 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:02.649 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:07.333 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:02.668 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:07.363 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:02.648 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:07.364 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:02.698 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:07.366 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:02.701 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:07.367 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:02.652 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:07.387 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:02.671 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:07.387 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:02.722 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:12.494 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:07.829 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:12.496 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:07.831 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:12.512 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:07.847 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:12.546 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:07.831 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:12.549 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:07.834 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:12.554 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:07.889 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:12.556 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:07.892 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:12.560 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:07.845 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:12.572 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:07.907 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:27.421 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:22.756 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:27.424 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:22.759 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:27.452 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:22.787 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:27.474 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:22.759 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:27.476 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:22.811 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:27.476 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:22.761 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:27.481 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:22.816 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:27.504 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:22.788 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:27.505 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:22.840 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:28.135 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:23.470 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:28.136 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:23.471 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:28.140 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:23.425 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:28.141 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:23.426 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:28.144 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:23.479 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:28.149 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:23.434 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:28.163 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:23.497 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:28.164 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:23.499 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:28.172 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:23.507 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:47.513 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:42.848 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:47.515 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:42.851 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:47.563 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:42.848 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:47.566 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:42.851 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:47.567 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:42.902 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:47.583 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:42.917 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:47.589 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:42.924 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:47.599 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:42.884 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:47.622 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:42.957 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:48.274 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:43.609 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:48.275 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:43.610 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:48.282 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:43.567 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:48.283 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:43.618 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:48.283 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:43.568 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:48.291 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:43.576 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:48.307 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:43.642 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:48.308 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:43.643 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:49:48.317 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:43.652 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:02.575 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:57.909 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:02.577 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:57.912 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:02.594 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:22:57.928 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:02.630 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:57.965 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:02.635 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:57.920 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:02.636 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:57.921 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:02.647 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:57.982 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:02.653 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:22:57.938 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:02.663 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:22:57.999 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:07.594 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:02.929 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:07.596 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:02.931 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:07.606 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:02.941 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:07.648 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:02.983 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:07.649 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:02.934 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:07.649 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:02.984 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:07.650 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:02.935 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:07.659 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:02.994 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:07.660 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:02.945 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:17.837 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:13.172 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:17.841 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:13.176 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:17.869 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:13.204 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:17.890 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:13.175 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:17.891 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:13.176 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:17.910 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:13.195 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:17.941 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:13.276 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:17.942 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:13.277 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:17.974 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:13.309 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:27.665 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:22.999 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:27.670 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:23.005 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:27.684 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:23.019 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:27.716 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:23.000 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:27.717 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:23.002 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:27.720 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:23.055 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:27.723 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:23.058 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:27.731 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:23.016 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:27.739 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:23.073 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:37.701 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:33.036 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:37.705 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:33.040 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:37.717 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:33.052 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:37.748 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:33.083 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:37.750 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:33.035 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:37.754 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:33.039 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:37.755 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:33.089 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:37.765 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:33.050 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:37.767 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:33.102 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:47.720 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:43.055 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:47.725 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:43.060 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:47.740 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:43.075 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:47.775 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:43.060 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:47.776 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:43.062 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:47.779 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:43.114 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:47.782 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:43.117 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:47.789 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:43.074 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:47.794 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:43.129 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:57.750 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:53.085 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:57.753 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:53.088 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:57.768 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:23:53.103 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:57.802 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:53.087 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:57.804 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:53.139 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:57.805 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:53.090 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:57.808 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:53.143 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:57.816 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:23:53.101 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:50:57.820 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:23:53.155 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:06.151 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:01.486 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:06.153 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:01.488 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:06.168 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:01.503 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:06.203 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:01.488 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:06.204 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:01.489 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:06.210 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:01.545 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:06.211 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:01.546 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:06.219 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:01.504 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:06.227 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:01.562 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:07.782 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:03.117 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:07.785 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:03.120 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:07.795 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:03.130 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:07.829 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:03.114 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:07.835 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:03.170 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:07.835 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:03.120 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:07.836 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:03.171 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:07.844 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:03.130 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:07.846 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:03.181 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:17.810 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:13.145 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:17.812 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:13.147 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:17.825 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:13.160 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:17.862 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:13.147 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:17.863 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:13.148 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:17.866 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:13.201 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:17.869 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:13.204 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:17.874 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:13.159 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:17.882 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:13.217 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:27.855 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:23.189 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:27.856 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:23.191 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:27.869 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:23.204 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:27.902 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:23.237 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:27.903 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:23.188 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:27.905 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:23.190 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:27.905 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:23.241 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:27.916 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:23.201 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:27.918 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:23.253 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:32.837 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:51:32.836+0000 12a61640 -1 osd.2 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:08.109341+0000 front 2023-07-29T13:50:08.110085+0000 (oldest deadline 2023-07-29T13:51:32.798751+0000) 2023-07-29T13:51:33.460 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:51:33.458+0000 12a61640 -1 osd.0 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:10.611868+0000 front 2023-07-29T13:50:10.613439+0000 (oldest deadline 2023-07-29T13:51:32.902955+0000) 2023-07-29T13:51:33.640 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:51:33.639+0000 1125e640 -1 osd.4 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:06.957553+0000 front 2023-07-29T13:50:06.959262+0000 (oldest deadline 2023-07-29T13:51:32.850441+0000) 2023-07-29T13:51:33.878 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:51:33.872+0000 12a61640 -1 osd.2 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:08.109341+0000 front 2023-07-29T13:50:08.110085+0000 (oldest deadline 2023-07-29T13:51:32.798751+0000) 2023-07-29T13:51:34.508 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:51:34.507+0000 12a61640 -1 osd.1 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:07.821318+0000 front 2023-07-29T13:50:07.823854+0000 (oldest deadline 2023-07-29T13:51:33.732137+0000) 2023-07-29T13:51:34.513 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:51:34.511+0000 12a61640 -1 osd.0 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:10.611868+0000 front 2023-07-29T13:50:10.613439+0000 (oldest deadline 2023-07-29T13:51:32.902955+0000) 2023-07-29T13:51:34.621 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:51:34.620+0000 1125e640 -1 osd.4 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:06.957553+0000 front 2023-07-29T13:50:06.959262+0000 (oldest deadline 2023-07-29T13:51:32.850441+0000) 2023-07-29T13:51:34.919 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:51:34.917+0000 12a61640 -1 osd.2 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:08.109341+0000 front 2023-07-29T13:50:08.110085+0000 (oldest deadline 2023-07-29T13:51:32.798751+0000) 2023-07-29T13:51:35.392 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:51:35.390+0000 12a61640 -1 osd.5 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:09.756552+0000 front 2023-07-29T13:50:09.751047+0000 (oldest deadline 2023-07-29T13:51:35.041848+0000) 2023-07-29T13:51:35.516 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:51:35.514+0000 12a61640 -1 osd.1 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:07.821318+0000 front 2023-07-29T13:50:07.823854+0000 (oldest deadline 2023-07-29T13:51:33.732137+0000) 2023-07-29T13:51:35.523 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:51:35.523+0000 12a61640 -1 osd.0 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:10.611868+0000 front 2023-07-29T13:50:10.613439+0000 (oldest deadline 2023-07-29T13:51:32.902955+0000) 2023-07-29T13:51:35.608 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:51:35.606+0000 1125e640 -1 osd.4 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:06.957553+0000 front 2023-07-29T13:50:06.959262+0000 (oldest deadline 2023-07-29T13:51:32.850441+0000) 2023-07-29T13:51:35.962 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:51:35.961+0000 12a61640 -1 osd.2 630 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:50:08.109341+0000 front 2023-07-29T13:50:08.110085+0000 (oldest deadline 2023-07-29T13:51:32.798751+0000) 2023-07-29T13:51:37.880 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:33.215 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:37.885 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:33.220 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:37.904 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:33.239 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:37.936 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:33.221 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:37.936 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:33.272 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:37.939 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:33.223 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:37.940 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:33.273 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:37.954 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:33.289 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:37.954 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:33.239 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:43.928 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:51:43.922+0000 2628e640 -1 snap_mapper.add_oid found existing snaps mapped on 58:8fe6bd9d:test-rados-api-smithi040-90886-76::foo:28, removing 2023-07-29T13:51:43.940 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:51:43.932+0000 29294640 -1 snap_mapper.add_oid found existing snaps mapped on 58:8fe6bd9d:test-rados-api-smithi040-90886-76::foo:28, removing 2023-07-29T13:51:45.663 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:51:45.662+0000 14264640 -1 osd.7 639 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:51:48.049 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:43.384 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:48.117 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:43.452 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:48.117 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:43.452 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:48.119 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:43.454 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:48.121 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:43.406 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:48.122 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:43.408 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:48.155 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:43.490 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:48.160 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:43.445 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:48.169 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:43.504 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:53.058 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:48.393 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:53.060 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:48.395 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:53.072 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:24:48.407 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:53.110 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:48.445 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:53.111 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:48.447 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:53.112 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:48.397 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:53.113 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:48.399 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:53.126 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:24:48.411 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:51:53.128 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:24:48.463 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:08.005 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:03.340 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:08.008 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:03.343 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:08.023 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:03.358 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:08.054 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:03.389 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:08.059 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:03.394 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:08.059 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:03.344 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:08.061 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:03.346 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:08.074 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:03.359 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:08.074 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:03.409 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:13.018 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:08.353 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:13.020 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:08.355 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:13.034 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:08.369 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:13.069 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:08.354 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:13.070 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:08.405 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:13.071 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:08.356 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:13.075 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:08.410 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:13.082 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:08.367 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:13.089 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:08.424 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:19.405 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:19.404+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:20.402 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:20.400+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:21.402 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:21.400+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:22.438 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:22.436+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:23.452 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:23.450+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:24.492 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:24.490+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:25.529 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:25.527+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:26.509 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:26.507+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:27.556 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:27.554+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:28.057 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:23.392 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.061 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:23.396 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.079 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:23.414 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.109 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:23.444 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.113 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:23.398 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.114 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:23.450 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.115 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:23.400 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.130 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:23.415 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.132 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:23.467 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.560 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:28.557+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:28.900 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:24.185 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.901 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:24.186 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.909 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:24.244 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.909 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:24.194 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.910 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:24.245 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.919 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:24.254 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.946 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:24.281 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.947 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:24.282 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:28.955 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:24.290 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:29.541 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:29.539+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:30.508 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:30.506+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:31.475 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:31.473+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:32.468 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:32.466+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:33.471 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:33.469+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:34.453 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:34.452+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:35.430 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:35.428+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:36.453 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:36.451+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:37.445 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:37.443+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:38.495 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:38.493+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:39.459 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:39.457+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:40.497 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:40.495+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:41.552 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:41.549+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:42.523 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:42.521+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:43.173 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:38.508 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:43.174 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:38.509 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:43.195 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:38.530 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:43.221 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:38.506 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:43.225 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:38.510 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:43.226 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:38.561 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:43.231 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:38.566 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:43.251 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:38.586 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:43.253 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:38.538 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:43.558 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:43.556+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:44.552 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:44.551+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:45.583 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:45.581+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:46.569 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:46.568+0000 12a61640 -1 osd.1 644 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:47.586 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:47.585+0000 12a61640 -1 osd.1 645 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:48.229 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:43.513 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:48.231 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:43.566 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:48.268 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:43.603 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:48.274 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:43.609 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:48.288 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:43.573 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:48.293 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:43.628 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:48.294 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:43.628 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:48.300 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:43.586 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:48.311 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:43.646 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:48.589 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:48.588+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:49.573 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:49.571+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:50.537 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:50.535+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:51.493 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:51.492+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:52.466 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:52.465+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:53.435 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:53.434+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:54.427 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:54.426+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:55.389 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:55.387+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:56.341 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:56.341+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:57.349 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:57.348+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:58.221 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:53.556 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:58.223 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:53.558 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:58.237 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:25:53.572 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:58.272 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:53.557 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:58.274 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:53.559 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:58.283 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:53.618 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:58.284 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:53.619 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:58.288 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:25:53.573 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:58.299 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:25:53.634 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:52:58.336 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:58.336+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:52:59.372 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:52:59.372+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:00.345 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:00.345+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:01.311 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:01.310+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:02.315 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:02.314+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:03.312 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:03.311+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:04.279 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:04.278+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:05.268 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:05.268+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:06.231 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:06.231+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:07.188 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:07.187+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:08.181 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:08.180+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:08.926 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:53:08.923+0000 12a61640 -1 osd.3 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:48.626286+0000 (oldest deadline 2023-07-29T13:53:08.626286+0000) 2023-07-29T13:53:09.147 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:09.147+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:09.793 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:53:09.792+0000 12a61640 -1 osd.0 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:51:47.950915+0000 front 2023-07-29T13:51:47.948412+0000 (oldest deadline 2023-07-29T13:53:09.037660+0000) 2023-07-29T13:53:09.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:53:09.944+0000 12a61640 -1 osd.3 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:48.626286+0000 (oldest deadline 2023-07-29T13:53:08.626286+0000) 2023-07-29T13:53:10.042 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:53:10.039+0000 12a61640 -1 osd.6 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:49.593162+0000 (oldest deadline 2023-07-29T13:53:09.593162+0000) 2023-07-29T13:53:10.147 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:10.146+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:10.542 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:53:10.534+0000 1125e640 -1 osd.4 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:51:47.521945+0000 front 2023-07-29T13:51:47.517685+0000 (oldest deadline 2023-07-29T13:53:09.714943+0000) 2023-07-29T13:53:10.732 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:53:10.728+0000 12a61640 -1 osd.2 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:50.446279+0000 (oldest deadline 2023-07-29T13:53:10.446279+0000) 2023-07-29T13:53:10.759 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:53:10.758+0000 12a61640 -1 osd.0 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:51:47.950915+0000 front 2023-07-29T13:51:47.948412+0000 (oldest deadline 2023-07-29T13:53:09.037660+0000) 2023-07-29T13:53:10.810 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:53:10.806+0000 12a61640 -1 osd.5 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:49.844834+0000 (oldest deadline 2023-07-29T13:53:09.844834+0000) 2023-07-29T13:53:10.949 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:53:10.947+0000 12a61640 -1 osd.3 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:48.626286+0000 (oldest deadline 2023-07-29T13:53:08.626286+0000) 2023-07-29T13:53:11.001 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:53:10.999+0000 12a61640 -1 osd.6 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:49.593162+0000 (oldest deadline 2023-07-29T13:53:09.593162+0000) 2023-07-29T13:53:11.161 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:11.157+0000 12a61640 -1 osd.1 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:51.079261+0000 (oldest deadline 2023-07-29T13:53:11.079261+0000) 2023-07-29T13:53:11.161 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:11.159+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:11.527 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:53:11.524+0000 1125e640 -1 osd.4 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:51:47.521945+0000 front 2023-07-29T13:51:47.517685+0000 (oldest deadline 2023-07-29T13:53:09.714943+0000) 2023-07-29T13:53:11.715 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:53:11.712+0000 12a61640 -1 osd.2 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:50.446279+0000 (oldest deadline 2023-07-29T13:53:10.446279+0000) 2023-07-29T13:53:11.801 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:53:11.799+0000 12a61640 -1 osd.0 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:51:47.950915+0000 front 2023-07-29T13:51:47.948412+0000 (oldest deadline 2023-07-29T13:53:09.037660+0000) 2023-07-29T13:53:11.827 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:53:11.825+0000 12a61640 -1 osd.5 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:49.844834+0000 (oldest deadline 2023-07-29T13:53:09.844834+0000) 2023-07-29T13:53:11.993 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:53:11.992+0000 12a61640 -1 osd.3 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:48.626286+0000 (oldest deadline 2023-07-29T13:53:08.626286+0000) 2023-07-29T13:53:12.015 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:53:12.014+0000 12a61640 -1 osd.6 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:49.593162+0000 (oldest deadline 2023-07-29T13:53:09.593162+0000) 2023-07-29T13:53:12.144 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:12.142+0000 12a61640 -1 osd.1 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:51.079261+0000 (oldest deadline 2023-07-29T13:53:11.079261+0000) 2023-07-29T13:53:12.145 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:12.143+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:12.490 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:53:12.488+0000 1125e640 -1 osd.4 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:51:47.521945+0000 front 2023-07-29T13:51:47.517685+0000 (oldest deadline 2023-07-29T13:53:09.714943+0000) 2023-07-29T13:53:12.699 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:53:12.697+0000 12a61640 -1 osd.2 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:50.446279+0000 (oldest deadline 2023-07-29T13:53:10.446279+0000) 2023-07-29T13:53:12.832 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:53:12.829+0000 12a61640 -1 osd.5 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:49.844834+0000 (oldest deadline 2023-07-29T13:53:09.844834+0000) 2023-07-29T13:53:12.846 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:53:12.844+0000 12a61640 -1 osd.0 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:51:47.950915+0000 front 2023-07-29T13:51:47.948412+0000 (oldest deadline 2023-07-29T13:53:09.037660+0000) 2023-07-29T13:53:13.030 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:53:13.020+0000 12a61640 -1 osd.3 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:48.626286+0000 (oldest deadline 2023-07-29T13:53:08.626286+0000) 2023-07-29T13:53:13.062 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:53:13.061+0000 12a61640 -1 osd.6 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:49.593162+0000 (oldest deadline 2023-07-29T13:53:09.593162+0000) 2023-07-29T13:53:13.129 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:13.127+0000 12a61640 -1 osd.1 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:51.079261+0000 (oldest deadline 2023-07-29T13:53:11.079261+0000) 2023-07-29T13:53:13.129 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:13.128+0000 12a61640 -1 osd.1 646 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:13.413 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:08.748 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:13.419 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:08.754 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:13.437 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:08.771 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:13.474 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:08.759 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:13.481 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:08.766 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:13.491 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:08.826 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:13.497 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:08.832 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:13.498 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:08.783 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:13.519 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:08.854 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:13.734 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:53:13.731+0000 12a61640 -1 osd.2 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:50.446279+0000 (oldest deadline 2023-07-29T13:53:10.446279+0000) 2023-07-29T13:53:13.802 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:53:13.798+0000 12a61640 -1 osd.0 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 since back 2023-07-29T13:51:47.950915+0000 front 2023-07-29T13:51:47.948412+0000 (oldest deadline 2023-07-29T13:53:09.037660+0000) 2023-07-29T13:53:13.994 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:53:13.992+0000 12a61640 -1 osd.3 646 heartbeat_check: no reply from 172.21.15.45:6806 osd.7 ever on either front or back, first ping sent 2023-07-29T13:51:48.626286+0000 (oldest deadline 2023-07-29T13:53:08.626286+0000) 2023-07-29T13:53:14.141 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:14.139+0000 12a61640 -1 osd.1 647 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4571.0:7146 218.8 218.eb1f2f08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e642) 2023-07-29T13:53:15.975 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:53:15.966+0000 2628e640 -1 snap_mapper.add_oid found existing snaps mapped on 58:ec1c8188:test-rados-api-smithi040-90886-78::foo:29, removing 2023-07-29T13:53:15.992 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:53:15.985+0000 302a2640 -1 snap_mapper.add_oid found existing snaps mapped on 58:ec1c8188:test-rados-api-smithi040-90886-78::foo:29, removing 2023-07-29T13:53:16.648 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:53:16.646+0000 14264640 -1 osd.7 649 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:53:18.491 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:13.826 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:18.502 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:13.837 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:18.537 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:13.872 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:18.561 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:13.846 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:18.564 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:13.849 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:18.568 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:13.903 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:18.571 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:13.906 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:18.590 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:13.875 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:18.594 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:13.929 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:28.497 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:23.832 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:28.502 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:23.837 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:28.522 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:23.857 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:28.553 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:23.888 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:28.555 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:23.840 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:28.557 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:23.892 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:28.558 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:23.843 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:28.578 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:23.863 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:28.579 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:23.914 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:33.532 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:28.867 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:33.536 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:28.871 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:33.548 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:28.884 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:33.586 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:28.871 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:33.590 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:28.875 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:33.601 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:28.886 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:33.607 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:28.942 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:33.608 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:28.944 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:33.618 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:28.953 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:39.160 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:34.495 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:39.162 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:34.497 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:39.177 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:34.512 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:39.214 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:34.499 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:39.216 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:34.501 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:39.216 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:34.551 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:39.218 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:34.553 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:39.230 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:34.515 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:39.232 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:34.567 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:43.267 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:53:43.255+0000 2628e640 -1 snap_mapper.add_oid found existing snaps mapped on 219:6b19f25c:test-rados-api-smithi040-90886-80::foo:2, removing 2023-07-29T13:53:43.280 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:53:43.270+0000 2e29e640 -1 snap_mapper.add_oid found existing snaps mapped on 219:6b19f25c:test-rados-api-smithi040-90886-80::foo:2, removing 2023-07-29T13:53:48.566 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:43.901 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:48.568 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:43.903 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:48.585 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:43.920 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:48.615 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:43.900 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:48.617 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:43.902 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:48.634 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:43.919 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:48.644 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:43.979 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:48.648 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:43.983 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:48.666 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:44.001 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:53.688 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:49.023 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:53.690 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:49.025 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:53.704 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:26:49.039 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:53.740 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:49.076 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:53.742 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:49.077 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:53.743 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:49.028 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:53.752 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:49.037 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:53.756 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:26:49.091 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:53:53.768 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:26:49.053 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:08.623 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:03.958 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:08.627 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:03.962 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:08.649 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:03.984 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:08.680 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:03.965 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:08.683 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:03.968 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:08.686 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:04.021 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:08.691 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:04.026 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:08.698 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:03.983 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:08.706 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:04.041 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:13.637 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:08.972 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:13.640 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:08.975 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:13.655 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:08.991 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:13.689 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:08.974 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:13.691 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:08.976 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:13.693 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:09.028 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:13.695 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:09.030 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:13.702 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:08.987 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:13.709 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:09.044 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:28.734 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:24.069 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:28.738 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:24.073 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:28.760 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:24.095 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:28.787 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:24.072 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:28.789 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:24.074 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:28.790 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:24.125 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:28.792 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:24.127 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:28.809 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:24.094 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:28.812 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:24.148 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:33.853 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:29.188 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:33.855 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:29.190 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:33.876 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:29.211 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:33.903 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:29.188 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:33.905 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:29.190 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:33.910 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:29.245 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:33.911 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:29.246 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:33.919 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:29.204 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:33.934 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:29.269 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:43.791 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:39.126 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:43.792 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:39.128 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:43.807 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:39.142 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:43.839 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:39.124 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:43.840 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:39.126 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:43.843 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:39.178 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:43.845 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:39.180 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:43.853 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:39.138 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:43.863 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:39.198 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:53.816 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:49.150 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:53.817 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:49.152 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:53.828 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:49.163 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:53.859 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:49.144 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:53.861 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:49.196 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:53.863 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:49.148 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:53.865 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:49.201 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:53.875 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:49.160 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:54:53.878 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:49.213 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:03.979 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:59.314 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:03.988 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:59.323 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:04.017 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:27:59.352 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:04.033 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:59.368 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:04.033 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:59.318 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:04.034 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:59.319 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:04.035 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:59.371 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:04.047 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:27:59.332 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:04.049 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:27:59.384 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:13.885 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:09.220 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:13.888 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:09.223 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:13.908 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:09.243 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:13.937 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:09.272 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:13.938 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:09.223 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:13.939 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:09.224 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:13.940 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:09.276 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:13.959 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:09.244 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:13.962 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:09.297 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:18.510 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ProxyRead (22425 ms) 2023-07-29T13:55:18.511 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2023-07-29T13:55:18.511 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (109643 ms) 2023-07-29T13:55:18.511 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2023-07-29T13:55:18.511 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (4460 ms) 2023-07-29T13:55:18.511 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2023-07-29T13:55:18.511 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.511 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3736 ms) 2023-07-29T13:55:18.512 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2023-07-29T13:55:18.512 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (19330 ms) 2023-07-29T13:55:18.512 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2023-07-29T13:55:18.512 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.512 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (18638 ms) 2023-07-29T13:55:18.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2023-07-29T13:55:18.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4948 ms) 2023-07-29T13:55:18.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2023-07-29T13:55:18.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (41526 ms) 2023-07-29T13:55:18.513 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2023-07-29T13:55:18.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (19016 ms) 2023-07-29T13:55:18.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2023-07-29T13:55:18.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (5647 ms) 2023-07-29T13:55:18.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2023-07-29T13:55:18.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.514 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (4404 ms) 2023-07-29T13:55:18.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2023-07-29T13:55:18.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (27147 ms) 2023-07-29T13:55:18.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2023-07-29T13:55:18.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (88258 ms) 2023-07-29T13:55:18.515 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2023-07-29T13:55:18.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (6699 ms) 2023-07-29T13:55:18.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2023-07-29T13:55:18.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (91518 ms) 2023-07-29T13:55:18.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2023-07-29T13:55:18.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.516 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: waiting for scrubs... 2023-07-29T13:55:18.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: done waiting 2023-07-29T13:55:18.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (27029 ms) 2023-07-29T13:55:18.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2023-07-29T13:55:18.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (42639 ms) 2023-07-29T13:55:18.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2023-07-29T13:55:18.517 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.518 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (38498 ms) 2023-07-29T13:55:18.518 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2023-07-29T13:55:18.518 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T13:55:18.518 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (10667 ms) 2023-07-29T13:55:18.518 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2023-07-29T13:55:18.982 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:14.317 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:19.034 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:14.369 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:19.051 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:14.386 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:19.087 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:14.422 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:19.091 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:14.426 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:19.100 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:14.385 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:19.106 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:14.391 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:19.109 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:14.444 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:19.120 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:14.405 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.028 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:29.363 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.030 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:29.365 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.057 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:29.392 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.081 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:29.366 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.081 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:29.416 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.083 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:29.368 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.083 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:29.418 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.107 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:29.392 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.117 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:29.453 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.757 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:30.042 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.758 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:30.043 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.763 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:30.098 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.764 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:30.099 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.766 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:30.051 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.769 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:30.104 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.770 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:30.105 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.773 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:30.108 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:34.779 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:30.114 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:54.083 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:49.418 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:54.085 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:49.420 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:54.102 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:49.436 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:54.137 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:49.421 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:54.141 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:49.424 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:54.141 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:49.474 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:54.142 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:49.476 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:54.157 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:49.442 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:54.158 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:49.493 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:59.093 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:54.428 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:59.094 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:54.429 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:59.105 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:28:54.440 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:59.142 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:54.427 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:59.143 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:54.429 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:59.145 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:54.481 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:59.147 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:54.482 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:59.154 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:28:54.440 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:55:59.157 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:28:54.492 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:14.138 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:09.473 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:14.141 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:09.476 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:14.163 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:09.498 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:14.191 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:09.476 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:14.192 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:09.477 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:14.202 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:09.537 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:14.203 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:09.538 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:14.205 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:09.491 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:14.218 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:09.553 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:16.508 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:11.843 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:16.509 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:11.844 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:16.523 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:11.858 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:16.560 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:11.845 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:16.561 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:11.846 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:16.574 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:11.859 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:16.593 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:11.928 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:16.594 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:11.930 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:16.609 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:11.944 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:19.152 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:14.487 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:19.154 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:14.489 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:19.164 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:14.499 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:19.203 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:14.488 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:19.204 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:14.489 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:19.205 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:14.541 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:19.207 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:14.542 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:19.213 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:14.498 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:19.219 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:14.555 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:34.191 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:29.526 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:34.196 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:29.531 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:34.215 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:29.550 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:34.242 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:29.527 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:34.246 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:29.531 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:34.269 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:29.554 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:34.270 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:29.605 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:34.273 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:29.608 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:34.294 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:29.625 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:39.207 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:34.542 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:39.210 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:34.545 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:39.220 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:34.555 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:39.258 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:34.543 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:39.260 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:34.545 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:39.260 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:34.596 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:39.262 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:34.597 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:39.273 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:34.558 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:39.275 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:34.610 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:41.466 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:56:41.458+0000 12a61640 -1 osd.1 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.998988+0000 front 2023-07-29T13:55:19.979962+0000 (oldest deadline 2023-07-29T13:56:40.996767+0000) 2023-07-29T13:56:41.616 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:56:41.615+0000 12a61640 -1 osd.0 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:15.735543+0000 front 2023-07-29T13:55:15.731339+0000 (oldest deadline 2023-07-29T13:56:41.020568+0000) 2023-07-29T13:56:42.467 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:56:42.466+0000 12a61640 -1 osd.2 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.409310+0000 front 2023-07-29T13:55:19.417233+0000 (oldest deadline 2023-07-29T13:56:42.301348+0000) 2023-07-29T13:56:42.484 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:56:42.482+0000 12a61640 -1 osd.1 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.998988+0000 front 2023-07-29T13:55:19.979962+0000 (oldest deadline 2023-07-29T13:56:40.996767+0000) 2023-07-29T13:56:42.601 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:56:42.600+0000 12a61640 -1 osd.0 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:15.735543+0000 front 2023-07-29T13:55:15.731339+0000 (oldest deadline 2023-07-29T13:56:41.020568+0000) 2023-07-29T13:56:43.498 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:56:43.495+0000 12a61640 -1 osd.2 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.409310+0000 front 2023-07-29T13:55:19.417233+0000 (oldest deadline 2023-07-29T13:56:42.301348+0000) 2023-07-29T13:56:43.532 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:56:43.531+0000 12a61640 -1 osd.1 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.998988+0000 front 2023-07-29T13:55:19.979962+0000 (oldest deadline 2023-07-29T13:56:40.996767+0000) 2023-07-29T13:56:43.631 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:56:43.630+0000 12a61640 -1 osd.0 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:15.735543+0000 front 2023-07-29T13:55:15.731339+0000 (oldest deadline 2023-07-29T13:56:41.020568+0000) 2023-07-29T13:56:44.052 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:56:44.050+0000 12a61640 -1 osd.3 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:20.544425+0000 front 2023-07-29T13:55:20.545727+0000 (oldest deadline 2023-07-29T13:56:43.435629+0000) 2023-07-29T13:56:44.489 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:56:44.487+0000 12a61640 -1 osd.2 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.409310+0000 front 2023-07-29T13:55:19.417233+0000 (oldest deadline 2023-07-29T13:56:42.301348+0000) 2023-07-29T13:56:44.558 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:56:44.556+0000 12a61640 -1 osd.1 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.998988+0000 front 2023-07-29T13:55:19.979962+0000 (oldest deadline 2023-07-29T13:56:40.996767+0000) 2023-07-29T13:56:44.623 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:56:44.622+0000 12a61640 -1 osd.0 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:15.735543+0000 front 2023-07-29T13:55:15.731339+0000 (oldest deadline 2023-07-29T13:56:41.020568+0000) 2023-07-29T13:56:44.898 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:56:44.896+0000 12a61640 -1 osd.6 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:18.344675+0000 front 2023-07-29T13:55:18.352512+0000 (oldest deadline 2023-07-29T13:56:44.238988+0000) 2023-07-29T13:56:45.078 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:56:45.077+0000 12a61640 -1 osd.3 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:20.544425+0000 front 2023-07-29T13:55:20.545727+0000 (oldest deadline 2023-07-29T13:56:43.435629+0000) 2023-07-29T13:56:45.420 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:56:45.417+0000 12a61640 -1 osd.7 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.985819+0000 front 2023-07-29T13:55:19.979137+0000 (oldest deadline 2023-07-29T13:56:44.587028+0000) 2023-07-29T13:56:45.423 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:56:45.420+0000 12a61640 -1 osd.7 698 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4180.0:54 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e698) 2023-07-29T13:56:45.534 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:56:45.532+0000 12a61640 -1 osd.2 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.409310+0000 front 2023-07-29T13:55:19.417233+0000 (oldest deadline 2023-07-29T13:56:42.301348+0000) 2023-07-29T13:56:45.586 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:56:45.584+0000 12a61640 -1 osd.0 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:15.735543+0000 front 2023-07-29T13:55:15.731339+0000 (oldest deadline 2023-07-29T13:56:41.020568+0000) 2023-07-29T13:56:45.594 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:56:45.592+0000 12a61640 -1 osd.1 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.998988+0000 front 2023-07-29T13:55:19.979962+0000 (oldest deadline 2023-07-29T13:56:40.996767+0000) 2023-07-29T13:56:45.877 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:56:45.874+0000 1125e640 -1 osd.4 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:20.020474+0000 front 2023-07-29T13:55:20.025626+0000 (oldest deadline 2023-07-29T13:56:45.459474+0000) 2023-07-29T13:56:45.903 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:56:45.901+0000 12a61640 -1 osd.6 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:18.344675+0000 front 2023-07-29T13:55:18.352512+0000 (oldest deadline 2023-07-29T13:56:44.238988+0000) 2023-07-29T13:56:46.116 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:56:46.114+0000 12a61640 -1 osd.3 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:20.544425+0000 front 2023-07-29T13:55:20.545727+0000 (oldest deadline 2023-07-29T13:56:43.435629+0000) 2023-07-29T13:56:46.391 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:56:46.386+0000 12a61640 -1 osd.7 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.985819+0000 front 2023-07-29T13:55:19.979137+0000 (oldest deadline 2023-07-29T13:56:44.587028+0000) 2023-07-29T13:56:46.392 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:56:46.390+0000 12a61640 -1 osd.7 698 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4180.0:54 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e698) 2023-07-29T13:56:46.520 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T13:56:46.518+0000 12a61640 -1 osd.2 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.409310+0000 front 2023-07-29T13:55:19.417233+0000 (oldest deadline 2023-07-29T13:56:42.301348+0000) 2023-07-29T13:56:46.552 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:56:46.551+0000 12a61640 -1 osd.1 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.998988+0000 front 2023-07-29T13:55:19.979962+0000 (oldest deadline 2023-07-29T13:56:40.996767+0000) 2023-07-29T13:56:46.613 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T13:56:46.611+0000 12a61640 -1 osd.0 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:15.735543+0000 front 2023-07-29T13:55:15.731339+0000 (oldest deadline 2023-07-29T13:56:41.020568+0000) 2023-07-29T13:56:46.894 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T13:56:46.892+0000 1125e640 -1 osd.4 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:20.020474+0000 front 2023-07-29T13:55:20.025626+0000 (oldest deadline 2023-07-29T13:56:45.459474+0000) 2023-07-29T13:56:46.913 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T13:56:46.911+0000 12a61640 -1 osd.6 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:18.344675+0000 front 2023-07-29T13:55:18.352512+0000 (oldest deadline 2023-07-29T13:56:44.238988+0000) 2023-07-29T13:56:47.118 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T13:56:47.116+0000 12a61640 -1 osd.3 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:20.544425+0000 front 2023-07-29T13:55:20.545727+0000 (oldest deadline 2023-07-29T13:56:43.435629+0000) 2023-07-29T13:56:47.386 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:56:47.383+0000 12a61640 -1 osd.7 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.985819+0000 front 2023-07-29T13:55:19.979137+0000 (oldest deadline 2023-07-29T13:56:44.587028+0000) 2023-07-29T13:56:47.389 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:56:47.387+0000 12a61640 -1 osd.7 698 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4180.0:54 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e698) 2023-07-29T13:56:47.543 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T13:56:47.541+0000 12a61640 -1 osd.1 698 heartbeat_check: no reply from 172.21.15.45:6820 osd.5 since back 2023-07-29T13:55:19.998988+0000 front 2023-07-29T13:55:19.979962+0000 (oldest deadline 2023-07-29T13:56:40.996767+0000) 2023-07-29T13:56:49.242 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:44.577 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:49.247 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:44.583 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:49.269 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:44.604 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:49.291 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:44.576 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:49.294 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:44.580 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:49.304 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:44.639 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:49.305 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:44.641 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:49.309 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:44.594 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:49.320 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:44.655 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:49.953 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:56:27.990+0000 15266640 -1 bdev(0x18ab56b0 /var/lib/ceph/osd/ceph-5/block) debug_aio_unlink stalled aio 0x312b4200 since 2023-07-29T13:55:29.978950+0000, timeout is 5s 2023-07-29T13:56:49.995 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:56:49.977+0000 12a61640 -1 osd.5 692 heartbeat_check: no reply from 172.21.15.40:6804 osd.2 since back 2023-07-29T13:55:16.599006+0000 front 2023-07-29T13:55:20.895069+0000 (oldest deadline 2023-07-29T13:56:40.730598+0000) 2023-07-29T13:56:49.997 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:56:49.979+0000 12a61640 -1 osd.5 692 heartbeat_check: no reply from 172.21.15.45:6830 osd.4 since back 2023-07-29T13:55:16.605723+0000 front 2023-07-29T13:55:16.600593+0000 (oldest deadline 2023-07-29T13:56:40.730598+0000) 2023-07-29T13:56:49.997 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:56:49.979+0000 12a61640 -1 osd.5 692 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T13:55:20.907654+0000 front 2023-07-29T13:55:16.601723+0000 (oldest deadline 2023-07-29T13:56:40.730598+0000) 2023-07-29T13:56:52.859 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:56:52.858+0000 14264640 -1 osd.5 702 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T13:56:59.354 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:54.689 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:59.356 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:54.691 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:59.386 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:54.721 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:59.408 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:54.743 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:59.410 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:54.745 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:59.412 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:54.698 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:59.414 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:54.699 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:59.437 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:54.722 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:56:59.438 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:54.773 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:04.296 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:59.631 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:04.300 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:59.635 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:04.316 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:29:59.651 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:04.349 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:59.634 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:04.350 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:59.636 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:04.354 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:59.690 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:04.358 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:59.693 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:04.364 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:29:59.649 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:04.374 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:29:59.710 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.339 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:14.674 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.341 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:14.677 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.373 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:14.708 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.390 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:14.675 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.392 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:14.677 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.403 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:14.738 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.404 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:14.740 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.412 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:14.697 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.424 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:14.759 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.984 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:15.319 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.985 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:15.320 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.987 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:15.273 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.988 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:15.274 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.993 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:15.328 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:19.996 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:15.281 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:20.004 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:15.339 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:20.006 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:15.341 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:20.017 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:15.353 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.445 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:34.781 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.449 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:34.785 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.480 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:34.815 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.499 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:34.784 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.502 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:34.838 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.503 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:34.788 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.505 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:34.841 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.534 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:34.869 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.540 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:34.825 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.977 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:35.312 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.978 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:35.314 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:39.987 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:35.322 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:40.020 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:35.355 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:40.021 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:35.306 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:40.021 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:35.356 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:40.022 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:35.307 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:40.030 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:35.315 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:40.032 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:35.367 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:49.234 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T13:57:49.224+0000 2628e640 -1 snap_mapper.add_oid found existing snaps mapped on 58:49b48f99:test-rados-api-smithi040-90886-97::foo:31, removing 2023-07-29T13:57:49.243 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T13:57:49.235+0000 29699640 -1 snap_mapper.add_oid found existing snaps mapped on 58:49b48f99:test-rados-api-smithi040-90886-97::foo:31, removing 2023-07-29T13:57:54.478 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:49.813 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:54.483 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:49.818 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:54.505 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:49.840 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:54.529 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:49.814 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:54.532 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:49.817 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:54.533 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:49.868 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:54.535 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:49.870 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:54.552 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:49.837 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:54.555 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:49.891 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:59.502 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:54.837 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:59.508 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:54.843 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:59.524 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:30:54.859 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:59.551 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:54.886 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:59.555 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:54.890 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:59.558 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:54.843 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:59.566 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:54.851 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:59.572 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:30:54.907 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:57:59.582 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:30:54.867 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:09.536 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:04.871 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:09.539 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:04.874 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:09.566 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:04.901 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:09.593 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:04.928 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:09.595 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:04.930 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:09.596 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:04.881 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:09.597 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:04.882 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:09.619 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:04.953 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:09.620 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:04.905 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:19.923 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:15.258 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:19.927 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:15.262 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:19.951 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:15.286 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:19.982 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:15.317 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:19.986 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:15.321 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:19.993 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:15.278 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:19.994 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:15.280 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:20.011 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:15.346 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:20.020 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:15.305 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:24.712 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:20.046 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:24.715 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:20.050 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:24.735 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:20.070 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:24.764 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:20.050 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:24.770 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:20.054 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:24.775 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:20.110 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:24.778 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:20.113 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:24.789 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:20.074 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:24.799 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:20.134 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:25.601 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:20.886 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:25.602 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:20.937 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:25.604 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:20.939 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:25.616 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:20.951 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:25.618 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:20.903 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:25.638 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:20.973 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:25.639 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:20.925 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:25.640 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:20.975 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:25.657 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:20.991 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:33.522 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:28.857 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:33.549 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:28.884 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:33.579 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:28.914 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:33.606 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:28.891 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:33.619 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:28.954 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:33.621 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:28.956 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:33.630 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:28.915 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:33.653 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:28.988 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:33.674 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:28.959 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:39.781 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:35.116 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:39.783 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:35.118 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:39.801 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:35.136 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:39.833 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:35.168 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:39.833 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:35.118 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:39.833 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:35.169 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:39.834 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:35.119 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:39.852 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:35.187 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:39.853 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:35.138 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:40.531 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:35.816 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:40.537 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:35.822 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:40.547 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:35.833 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:40.601 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:35.937 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:40.603 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:35.938 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:40.606 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:35.941 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:40.607 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:35.942 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:40.614 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:35.949 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:40.617 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:35.952 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:54.806 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:50.141 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:54.809 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:50.143 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:54.842 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:50.177 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:54.856 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:50.141 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:54.859 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:50.144 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:54.860 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:50.195 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:54.862 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:50.196 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:54.892 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:50.176 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:54.895 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:50.230 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:59.828 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:55.163 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:59.831 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:55.165 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:59.843 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:31:55.178 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:59.877 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:55.212 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:59.880 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:55.216 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:59.888 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:55.173 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:59.889 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:55.175 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:59.898 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:31:55.233 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:58:59.905 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:31:55.190 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:09.967 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:05.302 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:09.973 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:05.306 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:09.994 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:05.329 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:10.030 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:05.355 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:10.031 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:05.358 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:10.031 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:05.304 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:10.032 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:05.305 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:10.045 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:05.327 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:10.047 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:05.382 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:20.122 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:15.457 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:20.126 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:15.461 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:20.150 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:15.485 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:20.174 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:15.460 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:20.175 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:15.461 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:20.177 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:15.512 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:20.178 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:15.514 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:20.200 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:15.485 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:20.202 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:15.537 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:25.175 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:20.509 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:25.180 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:20.513 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:25.197 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:20.532 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:25.229 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:20.564 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:25.230 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:20.565 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:25.244 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:20.579 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:25.250 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:20.534 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:25.260 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:20.544 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:25.277 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:20.560 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:40.150 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:35.485 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:40.153 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:35.488 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:40.174 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:35.509 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:40.199 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:35.484 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:40.202 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:35.487 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:40.203 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:35.537 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:40.204 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:35.539 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:40.222 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:35.507 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:40.224 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:35.560 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:45.148 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:40.483 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:45.150 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:40.485 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:45.161 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:40.495 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:45.197 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:40.482 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:45.199 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:40.534 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:45.200 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:40.536 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:45.201 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:40.486 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:45.212 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:40.497 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T13:59:45.212 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:40.548 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:00.266 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:55.551 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:00.269 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:55.554 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:00.269 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:55.605 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:00.272 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:55.607 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:00.303 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:55.589 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:00.308 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:55.643 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:00.330 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:55.665 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:00.331 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:55.666 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:00.409 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:55.744 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:01.054 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:56.339 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:01.056 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:56.340 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:01.063 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:32:56.348 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:01.118 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:56.453 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:01.119 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:56.455 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:01.119 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:56.455 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:01.121 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:56.456 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:01.130 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:32:56.465 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:01.130 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:32:56.465 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:15.479 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:10.814 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:15.483 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:10.818 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:15.522 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:10.857 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:15.547 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:10.832 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:15.555 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:10.890 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:15.558 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:10.842 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:15.561 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:10.896 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:15.597 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:10.882 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:15.599 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:10.935 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:20.516 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:15.851 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:20.518 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:15.853 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:20.533 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:15.868 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:20.564 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:15.849 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:20.566 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:15.851 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:20.566 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:15.901 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:20.567 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:15.902 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:20.580 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:15.865 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:20.581 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:15.916 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:30.549 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:25.885 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:30.558 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:25.893 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:30.583 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:25.918 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:30.606 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:25.891 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:30.607 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:25.893 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:30.611 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:25.946 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:30.614 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:25.949 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:30.637 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:25.972 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:30.638 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:25.923 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:32.853 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:28.189 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:32.855 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:28.190 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:32.872 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:28.208 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:32.914 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:28.199 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:32.916 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:28.251 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:32.916 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:28.201 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:32.918 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:28.254 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:32.931 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:28.216 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:32.937 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:28.273 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:43.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T14:00:43.866 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (93892 ms) 2023-07-29T14:00:43.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2023-07-29T14:00:43.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T14:00:43.867 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (9020 ms) 2023-07-29T14:00:43.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2023-07-29T14:00:43.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T14:00:43.868 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (6653 ms) 2023-07-29T14:00:43.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2023-07-29T14:00:43.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T14:00:43.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (26760 ms) 2023-07-29T14:00:43.869 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2023-07-29T14:00:43.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T14:00:43.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (17963 ms) 2023-07-29T14:00:43.870 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2023-07-29T14:00:43.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (13568 ms) 2023-07-29T14:00:43.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2023-07-29T14:00:43.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2023-07-29T14:00:43.871 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: 00000010 69 73 |is| 2023-07-29T14:00:43.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: 00000012 2023-07-29T14:00:43.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12757 ms) 2023-07-29T14:00:43.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2023-07-29T14:00:43.872 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: require_osd_release = reef 2023-07-29T14:00:43.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (7130 ms) 2023-07-29T14:00:43.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (1674667 ms total) 2023-07-29T14:00:43.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: 2023-07-29T14:00:43.873 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2023-07-29T14:00:43.874 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2023-07-29T14:00:43.874 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1556 ms) 2023-07-29T14:00:43.874 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2023-07-29T14:00:43.874 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11203 ms) 2023-07-29T14:00:43.874 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2023-07-29T14:00:43.875 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (20090 ms) 2023-07-29T14:00:43.875 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2023-07-29T14:00:43.875 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (21 ms) 2023-07-29T14:00:43.875 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (32870 ms total) 2023-07-29T14:00:43.876 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: 2023-07-29T14:00:43.876 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2023-07-29T14:00:43.876 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2023-07-29T14:00:43.876 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7894 ms) 2023-07-29T14:00:43.877 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2023-07-29T14:00:43.877 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8856 ms) 2023-07-29T14:00:43.877 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2023-07-29T14:00:43.877 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: waiting for scrub... 2023-07-29T14:00:43.877 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: done waiting 2023-07-29T14:00:43.878 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (26719 ms) 2023-07-29T14:00:43.878 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2023-07-29T14:00:43.878 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (11959 ms) 2023-07-29T14:00:43.878 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2023-07-29T14:00:43.879 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (10028 ms) 2023-07-29T14:00:43.879 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2023-07-29T14:00:43.879 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8441 ms) 2023-07-29T14:00:43.879 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2023-07-29T14:00:43.880 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (11743 ms) 2023-07-29T14:00:43.880 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2023-07-29T14:00:45.792 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:41.128 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:45.797 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:41.132 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:45.854 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:41.139 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:45.860 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:41.145 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:45.869 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:41.204 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:45.873 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:41.208 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:45.904 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:33:41.239 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:45.918 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:33:41.203 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:00:45.924 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:33:41.259 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:05.785 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:01.120 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:05.787 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:01.123 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:05.819 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:01.154 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:05.842 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:01.127 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:05.844 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:01.129 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:05.847 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:01.182 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:05.848 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:01.184 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:05.877 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:01.163 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:05.888 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:01.223 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:20.731 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:16.066 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:20.733 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:16.068 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:20.774 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:16.109 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:20.786 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:16.070 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:20.789 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:16.074 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:20.790 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:16.124 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:20.791 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:16.126 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:20.817 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:16.102 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:20.827 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:16.163 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:25.725 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:21.060 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:25.727 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:21.062 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:25.740 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:21.075 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:25.776 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:21.061 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:25.777 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:21.063 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:25.788 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:21.073 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:25.789 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:21.125 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:25.793 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:21.128 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:25.803 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:21.138 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:40.777 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:36.112 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:40.779 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:36.115 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:40.782 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:36.067 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:40.803 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:36.138 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:40.809 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:36.094 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:40.826 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:36.110 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:40.833 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:36.168 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:40.836 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:36.171 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:40.856 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:36.189 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:55.850 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:51.176 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:55.851 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:51.178 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:55.872 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:34:51.205 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:55.890 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:51.226 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:55.893 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:51.228 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:55.895 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:51.180 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:55.898 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:51.181 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:55.927 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:34:51.210 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:01:55.934 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:34:51.269 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:06.070 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:01.406 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:06.072 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:01.408 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:06.101 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:01.436 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:06.128 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:01.411 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:06.136 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:01.419 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:06.138 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:01.473 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:06.140 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:01.476 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:06.158 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:01.443 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:06.166 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:01.501 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:10.938 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:06.273 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:10.994 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:06.329 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:11.011 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:06.346 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:11.046 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:06.381 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:11.049 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:06.384 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:11.053 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:06.338 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:11.062 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:06.346 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:11.064 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:06.399 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:11.074 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:06.360 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:26.061 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:21.396 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:26.064 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:21.399 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:26.093 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:21.428 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:26.115 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:21.400 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:26.116 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:21.402 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:26.119 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:21.455 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:26.122 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:21.458 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:26.145 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:21.430 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:26.152 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:21.488 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:27.024 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:22.359 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:27.033 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:22.361 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:27.048 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:22.377 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:27.049 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:22.381 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:27.049 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:22.382 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:27.060 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:22.345 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:27.061 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:22.346 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:27.070 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:22.356 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:27.076 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:22.411 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:40.820 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:36.155 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:40.829 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:36.164 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:40.876 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:36.161 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:40.877 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:36.212 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:40.884 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:36.219 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:40.889 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:36.224 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:40.899 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:36.183 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:40.927 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:36.212 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:40.946 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:36.281 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:41.755 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:37.040 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:41.762 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:37.047 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:41.773 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:37.058 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:41.782 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:37.117 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:41.793 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:37.121 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:41.809 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:37.140 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:41.850 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:37.186 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:41.853 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:37.188 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:41.865 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:37.200 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:46.279 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:41.614 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:46.285 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:41.620 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:46.307 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:35:41.635 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:46.336 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:41.671 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:46.338 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:41.673 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:46.339 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:41.624 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:46.341 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:41.626 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:46.350 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:35:41.685 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:02:46.356 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:35:41.641 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:06.473 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:01.808 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:06.478 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:01.813 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:06.514 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:01.848 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:06.531 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:01.866 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:06.535 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:01.870 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:06.536 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:01.821 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:06.542 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:01.825 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:06.579 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:01.864 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:06.585 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:01.917 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:11.411 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:06.744 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:11.411 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:06.746 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:11.425 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:06.760 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:11.456 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:06.789 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:11.457 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:06.792 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:11.463 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:06.748 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:11.464 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:06.749 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:11.473 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:06.808 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:11.477 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:06.763 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:26.550 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:21.885 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:26.609 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:21.894 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:26.610 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:21.895 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:26.615 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:21.948 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:26.641 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:21.970 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:26.641 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:21.919 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:26.674 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:22.005 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:26.674 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:22.008 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:26.691 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:22.026 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:31.487 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:26.821 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:31.492 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:26.826 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:31.508 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:26.843 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:31.540 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:26.874 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:31.541 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:26.826 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:31.549 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:26.831 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:31.549 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:26.878 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:31.560 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:26.895 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:31.563 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:26.848 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:46.537 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:41.872 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:46.540 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:41.874 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:46.562 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:41.898 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:46.585 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:41.920 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:46.591 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:41.876 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:46.591 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:41.923 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:46.594 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:41.879 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:46.615 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:41.950 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:46.621 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:41.906 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:51.510 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:46.846 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:51.571 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:46.906 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:51.579 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:46.864 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:51.588 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:46.873 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:51.603 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:36:46.888 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:51.620 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:36:46.955 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:51.621 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:46.957 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:51.623 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:46.959 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:03:51.640 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:36:46.976 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:06.623 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:01.949 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:06.623 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:01.953 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:06.651 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:01.985 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:06.666 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:01.951 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:06.680 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:01.955 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:06.681 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:02.007 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:06.681 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:02.009 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:06.703 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:01.986 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:06.723 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:02.053 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:07.465 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:02.751 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:07.470 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:02.752 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:07.478 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:02.760 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:07.491 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:02.826 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:07.493 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:02.828 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:07.505 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:02.839 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:07.570 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:02.905 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:07.571 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:02.906 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:07.580 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:02.915 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:26.673 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:22.008 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:26.679 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:22.014 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:26.704 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:22.039 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:26.735 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:22.069 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:26.736 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:22.017 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:26.736 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:22.019 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:26.738 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:22.074 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:26.755 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:22.040 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:26.759 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:22.095 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:27.529 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:22.813 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:27.529 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:22.814 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:27.541 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:22.822 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:27.576 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:22.911 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:27.576 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:22.911 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:27.577 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:22.912 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:27.577 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:22.912 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:27.586 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:22.921 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:27.586 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:22.921 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:46.756 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:42.090 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:46.760 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:42.093 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:46.782 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:42.112 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:46.810 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:42.144 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:46.811 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:42.096 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:46.813 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:42.148 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:46.814 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:42.099 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:46.833 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:42.118 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:46.836 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:42.171 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:47.606 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:42.891 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:47.607 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:42.892 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:47.619 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:37:42.900 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:47.628 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:42.960 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:47.628 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:42.961 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:47.637 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:37:42.972 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:47.680 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:43.009 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:47.680 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:43.011 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:04:47.687 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:37:43.022 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:06.810 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:02.146 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:06.812 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:02.147 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:06.832 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:02.165 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:06.860 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:02.193 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:06.861 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:02.143 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:06.861 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:02.144 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:06.862 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:02.197 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:06.876 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:02.161 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:06.881 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:02.214 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:07.655 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:02.940 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:07.660 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:02.942 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:07.665 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:02.949 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:07.740 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:03.074 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:07.740 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:03.075 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:07.741 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:03.076 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:07.742 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:03.077 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:07.749 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:03.084 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:07.751 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:03.086 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:11.915 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:07.250 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:11.920 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:07.252 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:11.935 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:07.270 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:11.969 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:07.304 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:11.973 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:07.308 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:11.986 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:07.272 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:11.989 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:07.274 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:11.994 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:07.329 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:12.005 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:07.290 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:24.698 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T14:05:24.697+0000 1125e640 -1 osd.4 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:01.130333+0000 front 2023-07-29T14:04:01.126828+0000 (oldest deadline 2023-07-29T14:05:24.622936+0000) 2023-07-29T14:05:24.812 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T14:05:24.811+0000 12a61640 -1 osd.6 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:03.117680+0000 front 2023-07-29T14:04:03.141131+0000 (oldest deadline 2023-07-29T14:05:24.785991+0000) 2023-07-29T14:05:25.689 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T14:05:25.688+0000 1125e640 -1 osd.4 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:01.130333+0000 front 2023-07-29T14:04:01.126828+0000 (oldest deadline 2023-07-29T14:05:24.622936+0000) 2023-07-29T14:05:25.722 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:05:25.721+0000 12a61640 -1 osd.7 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:00.093779+0000 front 2023-07-29T14:04:00.094979+0000 (oldest deadline 2023-07-29T14:05:24.786598+0000) 2023-07-29T14:05:25.791 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T14:05:25.790+0000 12a61640 -1 osd.6 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:03.117680+0000 front 2023-07-29T14:04:03.141131+0000 (oldest deadline 2023-07-29T14:05:24.785991+0000) 2023-07-29T14:05:26.026 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T14:05:26.024+0000 12a61640 -1 osd.3 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:02.317007+0000 front 2023-07-29T14:04:02.324410+0000 (oldest deadline 2023-07-29T14:05:25.231792+0000) 2023-07-29T14:05:26.079 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T14:05:26.077+0000 12a61640 -1 osd.5 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:02.248498+0000 front 2023-07-29T14:04:02.246284+0000 (oldest deadline 2023-07-29T14:05:25.191589+0000) 2023-07-29T14:05:26.718 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:05:26.716+0000 12a61640 -1 osd.7 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:00.093779+0000 front 2023-07-29T14:04:00.094979+0000 (oldest deadline 2023-07-29T14:05:24.786598+0000) 2023-07-29T14:05:26.728 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T14:05:26.726+0000 1125e640 -1 osd.4 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:01.130333+0000 front 2023-07-29T14:04:01.126828+0000 (oldest deadline 2023-07-29T14:05:24.622936+0000) 2023-07-29T14:05:26.783 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T14:05:26.782+0000 12a61640 -1 osd.6 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:03.117680+0000 front 2023-07-29T14:04:03.141131+0000 (oldest deadline 2023-07-29T14:05:24.785991+0000) 2023-07-29T14:05:26.802 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T14:05:26.800+0000 12a61640 -1 osd.2 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:04.191607+0000 front 2023-07-29T14:04:04.189400+0000 (oldest deadline 2023-07-29T14:05:25.885641+0000) 2023-07-29T14:05:26.901 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:22.236 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:26.907 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:22.241 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:26.923 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:22.258 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:26.957 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:22.292 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:26.958 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:22.243 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:26.968 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:22.295 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:26.969 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:22.245 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:26.974 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:22.259 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:26.981 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:22.317 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:27.027 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T14:05:27.025+0000 12a61640 -1 osd.3 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:02.317007+0000 front 2023-07-29T14:04:02.324410+0000 (oldest deadline 2023-07-29T14:05:25.231792+0000) 2023-07-29T14:05:27.055 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T14:05:27.048+0000 12a61640 -1 osd.5 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:02.248498+0000 front 2023-07-29T14:04:02.246284+0000 (oldest deadline 2023-07-29T14:05:25.191589+0000) 2023-07-29T14:05:27.692 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:05:27.691+0000 12a61640 -1 osd.7 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:00.093779+0000 front 2023-07-29T14:04:00.094979+0000 (oldest deadline 2023-07-29T14:05:24.786598+0000) 2023-07-29T14:05:27.693 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T14:05:27.692+0000 1125e640 -1 osd.4 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:01.130333+0000 front 2023-07-29T14:04:01.126828+0000 (oldest deadline 2023-07-29T14:05:24.622936+0000) 2023-07-29T14:05:27.707 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:22.991 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:27.708 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:22.992 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:27.721 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:23.000 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:27.789 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:23.123 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:27.789 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:23.124 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:27.797 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T14:05:27.795+0000 12a61640 -1 osd.2 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:04.191607+0000 front 2023-07-29T14:04:04.189400+0000 (oldest deadline 2023-07-29T14:05:25.885641+0000) 2023-07-29T14:05:27.805 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:23.141 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:27.807 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:23.142 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:27.808 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:23.144 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:27.811 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T14:05:27.809+0000 12a61640 -1 osd.6 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:03.117680+0000 front 2023-07-29T14:04:03.141131+0000 (oldest deadline 2023-07-29T14:05:24.785991+0000) 2023-07-29T14:05:27.817 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:23.152 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:27.990 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T14:05:27.989+0000 12a61640 -1 osd.3 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:02.317007+0000 front 2023-07-29T14:04:02.324410+0000 (oldest deadline 2023-07-29T14:05:25.231792+0000) 2023-07-29T14:05:28.071 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T14:05:28.069+0000 12a61640 -1 osd.5 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:02.248498+0000 front 2023-07-29T14:04:02.246284+0000 (oldest deadline 2023-07-29T14:05:25.191589+0000) 2023-07-29T14:05:28.194 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T14:05:28.193+0000 12a61640 -1 osd.1 994 heartbeat_check: no reply from 172.21.15.40:6812 osd.0 since back 2023-07-29T14:04:03.946677+0000 front 2023-07-29T14:04:03.944587+0000 (oldest deadline 2023-07-29T14:05:28.043146+0000) 2023-07-29T14:05:33.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T14:05:33.071+0000 15266640 -1 bdev(0x18e7f120 /var/lib/ceph/osd/ceph-0/block) debug_aio_unlink stalled aio 0x31e9ac30 since 2023-07-29T14:05:01.131233+0000, timeout is 5s 2023-07-29T14:05:35.320 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T14:05:35.318+0000 14264640 -1 osd.0 1000 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-29T14:05:36.963 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:32.298 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:36.966 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:32.301 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:36.968 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:32.253 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:36.992 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:32.327 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:37.021 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:32.306 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:37.023 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:32.359 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:37.025 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:32.360 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:37.050 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:32.335 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:37.053 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:32.388 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:38.371 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (9877 ms) 2023-07-29T14:05:38.371 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2023-07-29T14:05:38.372 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (14441 ms) 2023-07-29T14:05:38.372 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2023-07-29T14:05:38.372 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (11223 ms) 2023-07-29T14:05:38.372 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2023-07-29T14:05:38.372 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (39570 ms) 2023-07-29T14:05:38.372 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2023-07-29T14:05:38.372 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (9471 ms) 2023-07-29T14:05:38.373 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2023-07-29T14:05:38.373 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (9796 ms) 2023-07-29T14:05:38.373 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2023-07-29T14:05:38.373 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: hmm, no HitSet yet 2023-07-29T14:05:38.373 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: ok, hit_set contains 251:602f83fe:::foo:head 2023-07-29T14:05:38.373 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (10241 ms) 2023-07-29T14:05:38.373 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2023-07-29T14:05:38.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,0 2023-07-29T14:05:38.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: first is 1690639347 2023-07-29T14:05:38.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,0 2023-07-29T14:05:38.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,0 2023-07-29T14:05:38.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,1690639349,1690639350,0 2023-07-29T14:05:38.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,1690639349,1690639350,0 2023-07-29T14:05:38.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,1690639349,1690639350,0 2023-07-29T14:05:38.374 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,1690639349,1690639350,1690639353,1690639354,0 2023-07-29T14:05:38.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,1690639349,1690639350,1690639353,1690639354,0 2023-07-29T14:05:38.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,1690639349,1690639350,1690639353,1690639354,0 2023-07-29T14:05:38.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,1690639349,1690639350,1690639353,1690639354,1690639356,1690639357,0 2023-07-29T14:05:38.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,1690639349,1690639350,1690639353,1690639354,1690639356,1690639357,0 2023-07-29T14:05:38.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639347,1690639349,1690639350,1690639353,1690639354,1690639356,1690639357,0 2023-07-29T14:05:38.375 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: got ls 1690639350,1690639353,1690639354,1690639356,1690639357,1690639359,1690639360,0 2023-07-29T14:05:38.376 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: first now 1690639350, trimmed 2023-07-29T14:05:38.377 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (30992 ms) 2023-07-29T14:05:38.377 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2023-07-29T14:05:38.377 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: foo0 2023-07-29T14:05:38.377 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: verifying foo0 is eventually promoted 2023-07-29T14:05:38.377 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (18009 ms) 2023-07-29T14:05:38.378 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2023-07-29T14:05:38.378 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (20408 ms) 2023-07-29T14:05:38.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2023-07-29T14:05:38.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (24790 ms) 2023-07-29T14:05:38.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2023-07-29T14:05:38.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5431 ms) 2023-07-29T14:05:38.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2023-07-29T14:05:38.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (4680 ms) 2023-07-29T14:05:38.379 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2023-07-29T14:05:38.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (87872 ms) 2023-07-29T14:05:38.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2023-07-29T14:05:38.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (4408 ms) 2023-07-29T14:05:38.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (386849 ms total) 2023-07-29T14:05:38.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: 2023-07-29T14:05:38.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [----------] Global test environment tear-down 2023-07-29T14:05:38.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (2184658 ms total) 2023-07-29T14:05:38.380 INFO:tasks.workunit.client.0.smithi040.stdout: api_tier_pp: [ PASSED ] 77 tests. 2023-07-29T14:05:38.383 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.383 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90676 2023-07-29T14:05:38.383 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90676 2023-07-29T14:05:38.384 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.384 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90872 2023-07-29T14:05:38.384 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90872 2023-07-29T14:05:38.384 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.384 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91440 2023-07-29T14:05:38.384 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91440 2023-07-29T14:05:38.385 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.385 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91104 2023-07-29T14:05:38.385 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91104 2023-07-29T14:05:38.385 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.385 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91028 2023-07-29T14:05:38.385 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91028 2023-07-29T14:05:38.385 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.385 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90764 2023-07-29T14:05:38.386 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90764 2023-07-29T14:05:38.386 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.388 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91080 2023-07-29T14:05:38.388 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91080 2023-07-29T14:05:38.389 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.389 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90774 2023-07-29T14:05:38.389 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90774 2023-07-29T14:05:38.389 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.389 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91268 2023-07-29T14:05:38.389 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91268 2023-07-29T14:05:38.389 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.390 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90651 2023-07-29T14:05:38.390 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90651 2023-07-29T14:05:38.390 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.391 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90710 2023-07-29T14:05:38.391 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90710 2023-07-29T14:05:38.391 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.391 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91296 2023-07-29T14:05:38.391 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91296 2023-07-29T14:05:38.391 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.392 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90642 2023-07-29T14:05:38.392 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90642 2023-07-29T14:05:38.392 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.392 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90662 2023-07-29T14:05:38.392 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90662 2023-07-29T14:05:38.392 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.393 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90913 2023-07-29T14:05:38.393 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90913 2023-07-29T14:05:38.393 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.397 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91041 2023-07-29T14:05:38.398 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91041 2023-07-29T14:05:38.398 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.399 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90990 2023-07-29T14:05:38.399 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90990 2023-07-29T14:05:38.399 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.399 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91367 2023-07-29T14:05:38.399 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91367 2023-07-29T14:05:38.399 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.399 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91245 2023-07-29T14:05:38.399 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91245 2023-07-29T14:05:38.400 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.400 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90808 2023-07-29T14:05:38.400 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90808 2023-07-29T14:05:38.400 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.400 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91128 2023-07-29T14:05:38.400 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91128 2023-07-29T14:05:38.400 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.401 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90635 2023-07-29T14:05:38.401 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90635 2023-07-29T14:05:38.401 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.401 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90693 2023-07-29T14:05:38.401 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90693 2023-07-29T14:05:38.401 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.401 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91092 2023-07-29T14:05:38.402 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91092 2023-07-29T14:05:38.402 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.402 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=90736 2023-07-29T14:05:38.402 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 90736 2023-07-29T14:05:38.402 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.402 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91183 2023-07-29T14:05:38.402 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91183 2023-07-29T14:05:38.402 INFO:tasks.workunit.client.0.smithi040.stderr:+ for t in "${!pids[@]}" 2023-07-29T14:05:38.403 INFO:tasks.workunit.client.0.smithi040.stderr:+ pid=91214 2023-07-29T14:05:38.403 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 91214 2023-07-29T14:05:38.403 INFO:tasks.workunit.client.0.smithi040.stderr:+ exit 0 2023-07-29T14:05:38.403 INFO:tasks.workunit.client.0.smithi040.stderr:+ cleanup 2023-07-29T14:05:38.403 INFO:tasks.workunit.client.0.smithi040.stderr:+ pkill -P 90630 2023-07-29T14:05:38.404 INFO:tasks.workunit.client.0.smithi040.stderr:+ true 2023-07-29T14:05:38.404 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-07-29T14:05:38.405 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-07-29T14:05:38.441 INFO:tasks.workunit:Stopping ['rados/test.sh'] on client.0... 2023-07-29T14:05:38.442 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-07-29T14:05:38.774 DEBUG:teuthology.parallel:result is None 2023-07-29T14:05:38.775 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-07-29T14:05:38.809 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-07-29T14:05:38.809 DEBUG:teuthology.orchestra.run.smithi040:> rmdir -- /home/ubuntu/cephtest/mnt.0 2023-07-29T14:05:38.874 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2023-07-29T14:05:38.874 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-07-29T14:05:38.890 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-29T14:05:38.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-29T14:05:39.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.019+0000 7f7564a73640 10 monclient: get_monmap_and_config 2023-07-29T14:05:39.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.019+0000 7f7564a73640 10 monclient: build_initial_monmap 2023-07-29T14:05:39.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.019+0000 7f7564a73640 10 monclient: monmap: 2023-07-29T14:05:39.021 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:39.021 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:39.021 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:39.020893+0000 2023-07-29T14:05:39.021 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:39.020893+0000 2023-07-29T14:05:39.021 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:39.021 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:39.022 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.020+0000 7f7564a73640 1 Processor -- start 2023-07-29T14:05:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.020+0000 7f7564a73640 1 -- start start 2023-07-29T14:05:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.020+0000 7f7564a73640 10 monclient: init 2023-07-29T14:05:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.020+0000 7f7564a73640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.020+0000 7f7564a73640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2023-07-29T14:05:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.020+0000 7f7564a73640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.020+0000 7f7564a73640 10 monclient(hunting): picked mon.noname-b con 0x7f75600af800 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f7564a73640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f75600afbd0 0x7f756013e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f7564a73640 10 monclient(hunting): picked mon.noname-a con 0x7f75600afbd0 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f7564a73640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f75600aca90 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f7564a73640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7560071410 con 0x7f75600aca90 2023-07-29T14:05:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f7564a73640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f7564a73640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75600ac390 con 0x7f75600afbd0 2023-07-29T14:05:39.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f7564a73640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:39.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f7564a73640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75600ac670 con 0x7f75600af800 2023-07-29T14:05:39.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f7564a73640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:39.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f7564a73640 10 monclient(hunting): authenticate will time out at 3661.154135s 2023-07-29T14:05:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f755e59c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f7557fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f75600afbd0 0x7f756013e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f755e59c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600ab160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:37194/0 (socket says 172.21.15.40:37194) 2023-07-29T14:05:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f755e59c640 1 -- 172.21.15.40:0/1487212053 learned_addr learned my addr 172.21.15.40:0/1487212053 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:39.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f755e59c640 10 monclient(hunting): get_auth_request con 0x7f75600af800 auth_method 0 2023-07-29T14:05:39.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f755e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:39.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f755e59c640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:39.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.021+0000 7f755e59c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:39.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.022+0000 7f7557fff640 10 monclient(hunting): get_auth_request con 0x7f75600afbd0 auth_method 0 2023-07-29T14:05:39.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.022+0000 7f7557fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:39.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.022+0000 7f7557fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.022+0000 7f7557fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.022+0000 7f755e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.022+0000 7f755e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.022+0000 7f755e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.022+0000 7f7557fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.022+0000 7f7557fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.022+0000 7f7557fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.024+0000 7f755e59c640 10 monclient(hunting): handle_auth_done global_id 16334 payload 274 2023-07-29T14:05:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.024+0000 7f755e59c640 1 -- 172.21.15.40:0/1487212053 >> v2:172.21.15.40:3300/0 conn(0x7f75600afbd0 msgr2=0x7f756013e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:39.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.024+0000 7f755e59c640 1 --2- 172.21.15.40:0/1487212053 >> v2:172.21.15.40:3300/0 conn(0x7f75600afbd0 0x7f756013e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.024+0000 7f755e59c640 1 -- 172.21.15.40:0/1487212053 >> v1:172.21.15.40:6789/0 conn(0x7f75600aca90 legacy=0x7f7560070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:39.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.024+0000 7f755e59c640 10 monclient: _finish_hunting 0 2023-07-29T14:05:39.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.024+0000 7f755e59c640 1 monclient: found mon.noname-b 2023-07-29T14:05:39.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.024+0000 7f755e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:39.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.024+0000 7f755e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:39.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.024+0000 7f755e59c640 1 -- 172.21.15.40:0/1487212053 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75600aac40 con 0x7f75600af800 2023-07-29T14:05:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.026+0000 7f755e59c640 1 --2- 172.21.15.40:0/1487212053 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600ab160 secure :-1 s=READY pgs=3214 cs=0 l=1 rev1=1 crypto rx=0x7f7550009870 tx=0x7f755002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84494a65be43c513 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 1 -- 172.21.15.40:0/1487212053 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550031640 con 0x7f75600af800 2023-07-29T14:05:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 10 monclient: dump: 2023-07-29T14:05:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.036 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:39.036 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.036 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:39.036 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:39.036 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:39.036 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 10 monclient: _finish_auth 0 2023-07-29T14:05:39.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 10 monclient: _check_auth_tickets 2023-07-29T14:05:39.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:39.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 1 -- 172.21.15.40:0/1487212053 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75500317a0 con 0x7f75600af800 2023-07-29T14:05:39.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:39.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 1 -- 172.21.15.40:0/1487212053 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550031b30 con 0x7f75600af800 2023-07-29T14:05:39.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:39.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:39.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f755dd9b640 10 monclient: dump: 2023-07-29T14:05:39.038 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:39.038 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:39.039 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.039 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.039 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:39.039 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.039 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:39.039 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:39.039 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:39.039 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f7564a73640 5 monclient: authenticate success, global_id 16334 2023-07-29T14:05:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f7564a73640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f7564a73640 4 set_mon_vals no callback set 2023-07-29T14:05:39.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f7564a73640 10 monclient: shutdown 2023-07-29T14:05:39.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f7564a73640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:39.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f7564a73640 1 -- 172.21.15.40:0/1487212053 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 msgr2=0x7f75600ab160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:39.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.030+0000 7f7564a73640 1 --2- 172.21.15.40:0/1487212053 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600ab160 secure :-1 s=READY pgs=3214 cs=0 l=1 rev1=1 crypto rx=0x7f7550009870 tx=0x7f755002b8f0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.031+0000 7f7564a73640 1 -- 172.21.15.40:0/1487212053 shutdown_connections 2023-07-29T14:05:39.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.031+0000 7f7564a73640 1 --2- 172.21.15.40:0/1487212053 >> v2:172.21.15.40:3300/0 conn(0x7f75600afbd0 0x7f756013e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.031+0000 7f7564a73640 1 --2- 172.21.15.40:0/1487212053 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600ab160 unknown :-1 s=CLOSED pgs=3214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.031+0000 7f7564a73640 1 -- 172.21.15.40:0/1487212053 >> 172.21.15.40:0/1487212053 conn(0x7f7560065b00 msgr2=0x7f7560065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:39.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.031+0000 7f7564a73640 1 -- 172.21.15.40:0/1487212053 shutdown_connections 2023-07-29T14:05:39.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.031+0000 7f7564a73640 1 -- 172.21.15.40:0/1487212053 wait complete. 2023-07-29T14:05:39.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.032+0000 7f7564a73640 10 monclient: build_initial_monmap 2023-07-29T14:05:39.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.032+0000 7f7564a73640 10 monclient: monmap: 2023-07-29T14:05:39.056 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:39.056 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:39.057 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:39.057 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:39.057 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:39.057 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.058 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:39.058 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:39.058 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:39.058 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.032+0000 7f7564a73640 1 librados: starting msgr at 2023-07-29T14:05:39.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.032+0000 7f7564a73640 1 librados: starting objecter 2023-07-29T14:05:39.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.032+0000 7f7564a73640 1 Processor -- start 2023-07-29T14:05:39.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.032+0000 7f7564a73640 1 -- start start 2023-07-29T14:05:39.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.032+0000 7f7564a73640 1 librados: setting wanted keys 2023-07-29T14:05:39.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.032+0000 7f7564a73640 1 librados: calling monclient init 2023-07-29T14:05:39.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.032+0000 7f7564a73640 10 monclient: init 2023-07-29T14:05:39.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:39.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T14:05:39.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75600aca90 0x7f756013a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 10 monclient(hunting): picked mon.noname-b con 0x7f75600aca90 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:39.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600a9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 10 monclient(hunting): picked mon.noname-c con 0x7f75600af800 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:39.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75600afbd0 0x7f756013ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 10 monclient(hunting): picked mon.noname-a con 0x7f75600afbd0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:39.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:39.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756013b400 con 0x7f75600afbd0 2023-07-29T14:05:39.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7560132b10 con 0x7f75600aca90 2023-07-29T14:05:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7560132cd0 con 0x7f75600af800 2023-07-29T14:05:39.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:39.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7564a73640 10 monclient(hunting): authenticate will time out at 3661.166293s 2023-07-29T14:05:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7557fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f755ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75600afbd0 0x7f756013ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7557fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600a9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:37210/0 (socket says 172.21.15.40:37210) 2023-07-29T14:05:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f755e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75600aca90 0x7f756013a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7557fff640 1 -- 172.21.15.40:0/4082210353 learned_addr learned my addr 172.21.15.40:0/4082210353 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.033+0000 7f7557fff640 10 monclient(hunting): get_auth_request con 0x7f75600af800 auth_method 0 2023-07-29T14:05:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f7557fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:39.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f7557fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:39.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f7557fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:39.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755ed9d640 10 monclient(hunting): get_auth_request con 0x7f75600afbd0 auth_method 0 2023-07-29T14:05:39.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:39.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755ed9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:39.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755ed9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:39.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755e59c640 10 monclient(hunting): get_auth_request con 0x7f75600aca90 auth_method 0 2023-07-29T14:05:39.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:39.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755e59c640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:39.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755e59c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f7557fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f7557fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f7557fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.034+0000 7f755e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.037+0000 7f7557fff640 10 monclient(hunting): handle_auth_done global_id 16337 payload 824 2023-07-29T14:05:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.037+0000 7f7557fff640 1 -- 172.21.15.40:0/4082210353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75600aca90 msgr2=0x7f756013a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:39.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.037+0000 7f7557fff640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75600aca90 0x7f756013a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.037+0000 7f7557fff640 1 -- 172.21.15.40:0/4082210353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75600afbd0 msgr2=0x7f756013ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:39.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.037+0000 7f7557fff640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75600afbd0 0x7f756013ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.037+0000 7f7557fff640 10 monclient: _finish_hunting 0 2023-07-29T14:05:39.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.037+0000 7f7557fff640 1 monclient: found mon.noname-c 2023-07-29T14:05:39.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.037+0000 7f7557fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:39.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.037+0000 7f7557fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:39.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.037+0000 7f7557fff640 1 -- 172.21.15.40:0/4082210353 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500094d0 con 0x7f75600af800 2023-07-29T14:05:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.039+0000 7f7557fff640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600a9710 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 crypto rx=0x7f754800e840 tx=0x7f754800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.039+0000 7f7556ffd640 10 client.?.objecter ms_handle_connect 0x7f75600af800 2023-07-29T14:05:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.039+0000 7f7556ffd640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 1 -- 172.21.15.40:0/4082210353 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75480094f0 con 0x7f75600af800 2023-07-29T14:05:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:39.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:39.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 10 monclient: dump: 2023-07-29T14:05:39.082 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:39.082 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:39.082 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.083 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.083 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:39.083 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.083 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:39.084 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:39.084 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:39.084 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 10 monclient: _finish_auth 0 2023-07-29T14:05:39.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:39.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:39.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 1 -- 172.21.15.40:0/4082210353 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754800c940 con 0x7f75600af800 2023-07-29T14:05:39.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:39.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 1 -- 172.21.15.40:0/4082210353 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754800ccd0 con 0x7f75600af800 2023-07-29T14:05:39.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:39.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:39.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7556ffd640 10 monclient: dump: 2023-07-29T14:05:39.088 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:39.088 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:39.088 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.089 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.089 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:39.089 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.089 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:39.090 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:39.090 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:39.090 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7564a73640 5 monclient: authenticate success, global_id 16337 2023-07-29T14:05:39.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7564a73640 10 monclient: _renew_subs 2023-07-29T14:05:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7564a73640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f755d59a640 4 set_mon_vals no callback set 2023-07-29T14:05:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7564a73640 1 -- 172.21.15.40:0/4082210353 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7560132fb0 con 0x7f75600af800 2023-07-29T14:05:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7564a73640 10 client.16337.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7564a73640 10 monclient: _renew_subs 2023-07-29T14:05:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7564a73640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7564a73640 1 -- 172.21.15.40:0/4082210353 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7560133470 con 0x7f75600af800 2023-07-29T14:05:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.043+0000 7f7564a73640 1 librados: init done 2023-07-29T14:05:39.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.044+0000 7f7564a73640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:39.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.044+0000 7f7564a73640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:39.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.044+0000 7f7564a73640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:39.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.044+0000 7f7564a73640 1 -- 172.21.15.40:0/4082210353 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756013b570 con 0x7f75600af800 2023-07-29T14:05:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.301+0000 7f7556ffd640 1 -- 172.21.15.40:0/4082210353 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f7548013470 con 0x7f75600af800 2023-07-29T14:05:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.302+0000 7f7556ffd640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f754006bfb0 0x7f754006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.302+0000 7f755e59c640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f754006bfb0 0x7f754006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.303+0000 7f755e59c640 10 monclient: get_auth_request con 0x7f754006bfb0 auth_method 0 2023-07-29T14:05:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.303+0000 7f755e59c640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f754006bfb0 0x7f754006e460 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7550036000 tx=0x7f755003c040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.303+0000 7f7556ffd640 10 client.16337.objecter ms_handle_connect 0x7f754006bfb0 2023-07-29T14:05:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.314+0000 7f7556ffd640 1 -- 172.21.15.40:0/4082210353 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f754800ce30 con 0x7f75600af800 2023-07-29T14:05:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.314+0000 7f7556ffd640 10 client.16337.objecter ms_dispatch 0x7f7560000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.314+0000 7f7556ffd640 3 client.16337.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.314+0000 7f7556ffd640 3 client.16337.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.314+0000 7f7556ffd640 20 client.16337.objecter dump_active .. 0 homeless 2023-07-29T14:05:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.314+0000 7f7556ffd640 1 -- 172.21.15.40:0/4082210353 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f7548054770 con 0x7f75600af800 2023-07-29T14:05:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.314+0000 7f7556ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.314+0000 7f7556ffd640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:39.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.490+0000 7f7564a73640 1 -- 172.21.15.40:0/4082210353 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75600aa160 con 0x7f754006bfb0 2023-07-29T14:05:39.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.497+0000 7f7556ffd640 1 -- 172.21.15.40:0/4082210353 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+151950 (secure 0 0 0) 0x7f75600aa160 con 0x7f754006bfb0 2023-07-29T14:05:39.501 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T14:05:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-29T14:05:39.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 10 librados: watch_flush enter 2023-07-29T14:05:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 10 librados: watch_flush exit 2023-07-29T14:05:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 20 client.16337.objecter shutdown clearing up homeless session... 2023-07-29T14:05:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 10 client.16337.objecter successfully canceled tick 2023-07-29T14:05:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 -- 172.21.15.40:0/4082210353 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f754006bfb0 msgr2=0x7f754006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f754006bfb0 0x7f754006e460 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7550036000 tx=0x7f755003c040 comp rx=0 tx=0).stop 2023-07-29T14:05:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 10 monclient: shutdown 2023-07-29T14:05:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 -- 172.21.15.40:0/4082210353 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 msgr2=0x7f75600a9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600a9710 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 crypto rx=0x7f754800e840 tx=0x7f754800ed00 comp rx=0 tx=0).stop 2023-07-29T14:05:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 -- 172.21.15.40:0/4082210353 shutdown_connections 2023-07-29T14:05:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f754006bfb0 0x7f754006e460 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75600afbd0 0x7f756013ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f75600af800 0x7f75600a9710 unknown :-1 s=CLOSED pgs=3215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 --2- 172.21.15.40:0/4082210353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75600aca90 0x7f756013a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 -- 172.21.15.40:0/4082210353 >> 172.21.15.40:0/4082210353 conn(0x7f7560065b00 msgr2=0x7f75600ad210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:39.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 -- 172.21.15.40:0/4082210353 shutdown_connections 2023-07-29T14:05:39.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.498+0000 7f7564a73640 1 -- 172.21.15.40:0/4082210353 wait complete. 2023-07-29T14:05:39.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.499+0000 7f7564a73640 1 librados: shutdown 2023-07-29T14:05:39.532 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":2130,"stamp":"2023-07-29T14:05:38.312749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":525372,"kb_used_data":55548,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754449348,"statfs":{"total":773094113280,"available":772556132352,"internally_reserved":0,"allocated":56881152,"data_stored":31539903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22670,"apply_latency_ms":22670,"commit_latency_ns":22670000000,"apply_latency_ns":22670000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4000299,"num_objects":-44,"num_object_clones":-1,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44,"num_whiteouts":0,"num_read":-31,"num_read_kb":-11,"num_write":-71,"num_write_kb":-3969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.172818"},"pg_stats":[{"pgid":"7.1b","version":"0'0","reported_seq":968,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.762797+0000","last_change":"2023-07-29T13:56:55.662556+0000","last_active":"2023-07-29T14:05:36.762797+0000","last_peered":"2023-07-29T14:05:36.762797+0000","last_clean":"2023-07-29T14:05:36.762797+0000","last_became_active":"2023-07-29T13:56:55.656930+0000","last_became_peered":"2023-07-29T13:56:55.656930+0000","last_unstale":"2023-07-29T14:05:36.762797+0000","last_undegraded":"2023-07-29T14:05:36.762797+0000","last_fullsized":"2023-07-29T14:05:36.762797+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:52:51.321547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":977,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.016088+0000","last_change":"2023-07-29T13:41:38.328763+0000","last_active":"2023-07-29T14:05:37.016088+0000","last_peered":"2023-07-29T14:05:37.016088+0000","last_clean":"2023-07-29T14:05:37.016088+0000","last_became_active":"2023-07-29T13:41:38.322679+0000","last_became_peered":"2023-07-29T13:41:38.322679+0000","last_unstale":"2023-07-29T14:05:37.016088+0000","last_undegraded":"2023-07-29T14:05:37.016088+0000","last_fullsized":"2023-07-29T14:05:37.016088+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:40:55.886733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":964,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.997533+0000","last_change":"2023-07-29T13:29:52.105383+0000","last_active":"2023-07-29T14:05:36.997533+0000","last_peered":"2023-07-29T14:05:36.997533+0000","last_clean":"2023-07-29T14:05:36.997533+0000","last_became_active":"2023-07-29T13:29:52.099163+0000","last_became_peered":"2023-07-29T13:29:52.099163+0000","last_unstale":"2023-07-29T14:05:36.997533+0000","last_undegraded":"2023-07-29T14:05:36.997533+0000","last_fullsized":"2023-07-29T14:05:36.997533+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:03:36.535099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":930,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.709810+0000","last_change":"2023-07-29T13:29:41.816664+0000","last_active":"2023-07-29T14:05:36.709810+0000","last_peered":"2023-07-29T14:05:36.709810+0000","last_clean":"2023-07-29T14:05:36.709810+0000","last_became_active":"2023-07-29T13:29:41.782506+0000","last_became_peered":"2023-07-29T13:29:41.782506+0000","last_unstale":"2023-07-29T14:05:36.709810+0000","last_undegraded":"2023-07-29T14:05:36.709810+0000","last_fullsized":"2023-07-29T14:05:36.709810+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:20:57.566653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":17,"reported_epoch":1000,"state":"active+undersized","last_fresh":"2023-07-29T14:05:34.211410+0000","last_change":"2023-07-29T14:05:29.622538+0000","last_active":"2023-07-29T14:05:34.211410+0000","last_peered":"2023-07-29T14:05:34.211410+0000","last_clean":"2023-07-29T13:29:15.987740+0000","last_became_active":"2023-07-29T14:05:29.622538+0000","last_became_peered":"2023-07-29T14:05:29.622538+0000","last_unstale":"2023-07-29T14:05:34.211410+0000","last_undegraded":"2023-07-29T14:05:34.211410+0000","last_fullsized":"2023-07-29T14:05:29.578919+0000","mapping_epoch":995,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:53:28.485630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":910,"reported_epoch":999,"state":"active+clean","last_fresh":"2023-07-29T14:05:33.122660+0000","last_change":"2023-07-29T13:53:19.606247+0000","last_active":"2023-07-29T14:05:33.122660+0000","last_peered":"2023-07-29T14:05:33.122660+0000","last_clean":"2023-07-29T14:05:33.122660+0000","last_became_active":"2023-07-29T13:53:19.600386+0000","last_became_peered":"2023-07-29T13:53:19.600386+0000","last_unstale":"2023-07-29T14:05:33.122660+0000","last_undegraded":"2023-07-29T14:05:33.122660+0000","last_fullsized":"2023-07-29T14:05:33.122660+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:46:03.055229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":934,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.436676+0000","last_change":"2023-07-29T13:56:55.406251+0000","last_active":"2023-07-29T14:05:34.436676+0000","last_peered":"2023-07-29T14:05:34.436676+0000","last_clean":"2023-07-29T14:05:34.436676+0000","last_became_active":"2023-07-29T13:56:55.400591+0000","last_became_peered":"2023-07-29T13:56:55.400591+0000","last_unstale":"2023-07-29T14:05:34.436676+0000","last_undegraded":"2023-07-29T14:05:34.436676+0000","last_fullsized":"2023-07-29T14:05:34.436676+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:40:48.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":954,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.369590+0000","last_change":"2023-07-29T13:29:31.089676+0000","last_active":"2023-07-29T14:05:36.369590+0000","last_peered":"2023-07-29T14:05:36.369590+0000","last_clean":"2023-07-29T14:05:36.369590+0000","last_became_active":"2023-07-29T13:29:31.072042+0000","last_became_peered":"2023-07-29T13:29:31.072042+0000","last_unstale":"2023-07-29T14:05:36.369590+0000","last_undegraded":"2023-07-29T14:05:36.369590+0000","last_fullsized":"2023-07-29T14:05:36.369590+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:28:28.141003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":909,"reported_epoch":999,"state":"active+clean","last_fresh":"2023-07-29T14:05:33.092168+0000","last_change":"2023-07-29T13:53:19.677067+0000","last_active":"2023-07-29T14:05:33.092168+0000","last_peered":"2023-07-29T14:05:33.092168+0000","last_clean":"2023-07-29T14:05:33.092168+0000","last_became_active":"2023-07-29T13:53:19.671215+0000","last_became_peered":"2023-07-29T13:53:19.671215+0000","last_unstale":"2023-07-29T14:05:33.092168+0000","last_undegraded":"2023-07-29T14:05:33.092168+0000","last_fullsized":"2023-07-29T14:05:33.092168+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:51:18.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":985,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.445062+0000","last_change":"2023-07-29T13:56:55.517147+0000","last_active":"2023-07-29T14:05:35.445062+0000","last_peered":"2023-07-29T14:05:35.445062+0000","last_clean":"2023-07-29T14:05:35.445062+0000","last_became_active":"2023-07-29T13:56:55.511490+0000","last_became_peered":"2023-07-29T13:56:55.511490+0000","last_unstale":"2023-07-29T14:05:35.445062+0000","last_undegraded":"2023-07-29T14:05:35.445062+0000","last_fullsized":"2023-07-29T14:05:35.445062+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:26.433868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":962,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.507347+0000","last_change":"2023-07-29T14:05:37.515837+0000","last_active":"2023-07-29T14:05:36.327038+0000","last_peered":"2023-07-29T14:05:35.596454+0000","last_clean":"2023-07-29T14:04:22.472067+0000","last_became_active":"2023-07-29T14:05:29.733569+0000","last_became_peered":"2023-07-29T14:05:29.733569+0000","last_unstale":"2023-07-29T14:05:37.507347+0000","last_undegraded":"2023-07-29T14:05:37.507347+0000","last_fullsized":"2023-07-29T14:05:37.507347+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:03:53.476296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":909,"reported_epoch":999,"state":"active+clean","last_fresh":"2023-07-29T14:05:33.095461+0000","last_change":"2023-07-29T13:53:19.644608+0000","last_active":"2023-07-29T14:05:33.095461+0000","last_peered":"2023-07-29T14:05:33.095461+0000","last_clean":"2023-07-29T14:05:33.095461+0000","last_became_active":"2023-07-29T13:53:19.637473+0000","last_became_peered":"2023-07-29T13:53:19.637473+0000","last_unstale":"2023-07-29T14:05:33.095461+0000","last_undegraded":"2023-07-29T14:05:33.095461+0000","last_fullsized":"2023-07-29T14:05:33.095461+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:53:02.384899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":932,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.746967+0000","last_change":"2023-07-29T13:29:33.530600+0000","last_active":"2023-07-29T14:05:36.746967+0000","last_peered":"2023-07-29T14:05:36.746967+0000","last_clean":"2023-07-29T14:05:36.746967+0000","last_became_active":"2023-07-29T13:29:33.515255+0000","last_became_peered":"2023-07-29T13:29:33.515255+0000","last_unstale":"2023-07-29T14:05:36.746967+0000","last_undegraded":"2023-07-29T14:05:36.746967+0000","last_fullsized":"2023-07-29T14:05:36.746967+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:21:56.794070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":979,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.433490+0000","last_change":"2023-07-29T14:05:37.441418+0000","last_active":"2023-07-29T14:05:36.969928+0000","last_peered":"2023-07-29T14:05:35.332022+0000","last_clean":"2023-07-29T14:05:12.448301+0000","last_became_active":"2023-07-29T14:05:29.733673+0000","last_became_peered":"2023-07-29T14:05:29.733673+0000","last_unstale":"2023-07-29T14:05:37.433490+0000","last_undegraded":"2023-07-29T14:05:37.433490+0000","last_fullsized":"2023-07-29T14:05:37.433490+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:38:04.875711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":936,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.418624+0000","last_change":"2023-07-29T13:56:55.146603+0000","last_active":"2023-07-29T14:05:34.418624+0000","last_peered":"2023-07-29T14:05:34.418624+0000","last_clean":"2023-07-29T14:05:34.418624+0000","last_became_active":"2023-07-29T13:56:55.139167+0000","last_became_peered":"2023-07-29T13:56:55.139167+0000","last_unstale":"2023-07-29T14:05:34.418624+0000","last_undegraded":"2023-07-29T14:05:34.418624+0000","last_fullsized":"2023-07-29T14:05:34.418624+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:56:23.477235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":944,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.533539+0000","last_change":"2023-07-29T14:05:37.544652+0000","last_active":"2023-07-29T14:05:36.720848+0000","last_peered":"2023-07-29T14:05:35.604442+0000","last_clean":"2023-07-29T14:04:06.556968+0000","last_became_active":"2023-07-29T14:05:29.660537+0000","last_became_peered":"2023-07-29T14:05:29.660537+0000","last_unstale":"2023-07-29T14:05:37.533539+0000","last_undegraded":"2023-07-29T14:05:37.533539+0000","last_fullsized":"2023-07-29T14:05:37.533539+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:42:16.790121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":987,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.436816+0000","last_change":"2023-07-29T13:56:55.073408+0000","last_active":"2023-07-29T14:05:35.436816+0000","last_peered":"2023-07-29T14:05:35.436816+0000","last_clean":"2023-07-29T14:05:35.436816+0000","last_became_active":"2023-07-29T13:56:55.065724+0000","last_became_peered":"2023-07-29T13:56:55.065724+0000","last_unstale":"2023-07-29T14:05:35.436816+0000","last_undegraded":"2023-07-29T14:05:35.436816+0000","last_fullsized":"2023-07-29T14:05:35.436816+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:42:58.825948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":971,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.050103+0000","last_change":"2023-07-29T13:28:14.434312+0000","last_active":"2023-07-29T14:05:37.050103+0000","last_peered":"2023-07-29T14:05:37.050103+0000","last_clean":"2023-07-29T14:05:37.050103+0000","last_became_active":"2023-07-29T13:28:14.423044+0000","last_became_peered":"2023-07-29T13:28:14.423044+0000","last_unstale":"2023-07-29T14:05:37.050103+0000","last_undegraded":"2023-07-29T14:05:37.050103+0000","last_fullsized":"2023-07-29T14:05:37.050103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:56:27.865715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1","version":"70'7","reported_seq":983,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.310515+0000","last_change":"2023-07-29T13:41:38.528285+0000","last_active":"2023-07-29T14:05:36.310515+0000","last_peered":"2023-07-29T14:05:36.310515+0000","last_clean":"2023-07-29T14:05:36.310515+0000","last_became_active":"2023-07-29T13:41:38.521847+0000","last_became_peered":"2023-07-29T13:41:38.521847+0000","last_unstale":"2023-07-29T14:05:36.310515+0000","last_undegraded":"2023-07-29T14:05:36.310515+0000","last_fullsized":"2023-07-29T14:05:36.310515+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:08:42.836378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":942,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.430995+0000","last_change":"2023-07-29T13:56:55.263959+0000","last_active":"2023-07-29T14:05:34.430995+0000","last_peered":"2023-07-29T14:05:34.430995+0000","last_clean":"2023-07-29T14:05:34.430995+0000","last_became_active":"2023-07-29T13:56:55.258241+0000","last_became_peered":"2023-07-29T13:56:55.258241+0000","last_unstale":"2023-07-29T14:05:34.430995+0000","last_undegraded":"2023-07-29T14:05:34.430995+0000","last_fullsized":"2023-07-29T14:05:34.430995+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:42.517469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":966,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.065287+0000","last_change":"2023-07-29T13:29:52.230014+0000","last_active":"2023-07-29T14:05:37.065287+0000","last_peered":"2023-07-29T14:05:37.065287+0000","last_clean":"2023-07-29T14:05:37.065287+0000","last_became_active":"2023-07-29T13:29:52.224467+0000","last_became_peered":"2023-07-29T13:29:52.224467+0000","last_unstale":"2023-07-29T14:05:37.065287+0000","last_undegraded":"2023-07-29T14:05:37.065287+0000","last_fullsized":"2023-07-29T14:05:37.065287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:51:03.019114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":972,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.014430+0000","last_change":"2023-07-29T13:28:14.652586+0000","last_active":"2023-07-29T14:05:37.014430+0000","last_peered":"2023-07-29T14:05:37.014430+0000","last_clean":"2023-07-29T14:05:37.014430+0000","last_became_active":"2023-07-29T13:28:14.642023+0000","last_became_peered":"2023-07-29T13:28:14.642023+0000","last_unstale":"2023-07-29T14:05:37.014430+0000","last_undegraded":"2023-07-29T14:05:37.014430+0000","last_fullsized":"2023-07-29T14:05:37.014430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:25:23.334956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":933,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.764552+0000","last_change":"2023-07-29T13:29:41.869711+0000","last_active":"2023-07-29T14:05:36.764552+0000","last_peered":"2023-07-29T14:05:36.764552+0000","last_clean":"2023-07-29T14:05:36.764552+0000","last_became_active":"2023-07-29T13:29:41.844314+0000","last_became_peered":"2023-07-29T13:29:41.844314+0000","last_unstale":"2023-07-29T14:05:36.764552+0000","last_undegraded":"2023-07-29T14:05:36.764552+0000","last_fullsized":"2023-07-29T14:05:36.764552+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:41:40.503015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":960,"reported_epoch":999,"state":"active+clean","last_fresh":"2023-07-29T14:05:33.082162+0000","last_change":"2023-07-29T13:56:55.159956+0000","last_active":"2023-07-29T14:05:33.082162+0000","last_peered":"2023-07-29T14:05:33.082162+0000","last_clean":"2023-07-29T14:05:33.082162+0000","last_became_active":"2023-07-29T13:56:55.150514+0000","last_became_peered":"2023-07-29T13:56:55.150514+0000","last_unstale":"2023-07-29T14:05:33.082162+0000","last_undegraded":"2023-07-29T14:05:33.082162+0000","last_fullsized":"2023-07-29T14:05:33.082162+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:16:38.329000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1005,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.048452+0000","last_change":"2023-07-29T13:56:55.615681+0000","last_active":"2023-07-29T14:05:37.048452+0000","last_peered":"2023-07-29T14:05:37.048452+0000","last_clean":"2023-07-29T14:05:37.048452+0000","last_became_active":"2023-07-29T13:56:55.609414+0000","last_became_peered":"2023-07-29T13:56:55.609414+0000","last_unstale":"2023-07-29T14:05:37.048452+0000","last_undegraded":"2023-07-29T14:05:37.048452+0000","last_fullsized":"2023-07-29T14:05:37.048452+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:36:30.601005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1012,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.999968+0000","last_change":"2023-07-29T13:56:55.550112+0000","last_active":"2023-07-29T14:05:36.999968+0000","last_peered":"2023-07-29T14:05:36.999968+0000","last_clean":"2023-07-29T14:05:36.999968+0000","last_became_active":"2023-07-29T13:56:55.543861+0000","last_became_peered":"2023-07-29T13:56:55.543861+0000","last_unstale":"2023-07-29T14:05:36.999968+0000","last_undegraded":"2023-07-29T14:05:36.999968+0000","last_fullsized":"2023-07-29T14:05:36.999968+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:57:36.179121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":940,"reported_epoch":1000,"state":"active+undersized","last_fresh":"2023-07-29T14:05:34.441767+0000","last_change":"2023-07-29T14:05:29.759408+0000","last_active":"2023-07-29T14:05:34.441767+0000","last_peered":"2023-07-29T14:05:34.441767+0000","last_clean":"2023-07-29T14:04:06.482879+0000","last_became_active":"2023-07-29T14:05:29.759408+0000","last_became_peered":"2023-07-29T14:05:29.759408+0000","last_unstale":"2023-07-29T14:05:34.441767+0000","last_undegraded":"2023-07-29T14:05:34.441767+0000","last_fullsized":"2023-07-29T14:05:29.709468+0000","mapping_epoch":995,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:11:56.037546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":972,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.010320+0000","last_change":"2023-07-29T13:28:14.713485+0000","last_active":"2023-07-29T14:05:37.010320+0000","last_peered":"2023-07-29T14:05:37.010320+0000","last_clean":"2023-07-29T14:05:37.010320+0000","last_became_active":"2023-07-29T13:28:14.708042+0000","last_became_peered":"2023-07-29T13:28:14.708042+0000","last_unstale":"2023-07-29T14:05:37.010320+0000","last_undegraded":"2023-07-29T14:05:37.010320+0000","last_fullsized":"2023-07-29T14:05:37.010320+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:25:56.525512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":1099,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.438875+0000","last_change":"2023-07-29T13:56:55.164457+0000","last_active":"2023-07-29T14:05:35.438875+0000","last_peered":"2023-07-29T14:05:35.438875+0000","last_clean":"2023-07-29T14:05:35.438875+0000","last_became_active":"2023-07-29T13:56:55.157702+0000","last_became_peered":"2023-07-29T13:56:55.157702+0000","last_unstale":"2023-07-29T14:05:35.438875+0000","last_undegraded":"2023-07-29T14:05:35.438875+0000","last_fullsized":"2023-07-29T14:05:35.438875+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_clean_scrub_stamp":"2023-07-29T13:28:06.434639+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:06:43.352094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":995,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.367084+0000","last_change":"2023-07-29T13:56:55.347022+0000","last_active":"2023-07-29T14:05:36.367084+0000","last_peered":"2023-07-29T14:05:36.367084+0000","last_clean":"2023-07-29T14:05:36.367084+0000","last_became_active":"2023-07-29T13:56:55.341426+0000","last_became_peered":"2023-07-29T13:56:55.341426+0000","last_unstale":"2023-07-29T14:05:36.367084+0000","last_undegraded":"2023-07-29T14:05:36.367084+0000","last_fullsized":"2023-07-29T14:05:36.367084+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:46:20.874037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":945,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.466789+0000","last_change":"2023-07-29T13:41:38.063359+0000","last_active":"2023-07-29T14:05:35.466789+0000","last_peered":"2023-07-29T14:05:35.466789+0000","last_clean":"2023-07-29T14:05:35.466789+0000","last_became_active":"2023-07-29T13:41:38.057649+0000","last_became_peered":"2023-07-29T13:41:38.057649+0000","last_unstale":"2023-07-29T14:05:35.466789+0000","last_undegraded":"2023-07-29T14:05:35.466789+0000","last_fullsized":"2023-07-29T14:05:35.466789+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:36:46.868780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":922,"reported_epoch":999,"state":"active+clean","last_fresh":"2023-07-29T14:05:33.125606+0000","last_change":"2023-07-29T13:53:19.566408+0000","last_active":"2023-07-29T14:05:33.125606+0000","last_peered":"2023-07-29T14:05:33.125606+0000","last_clean":"2023-07-29T14:05:33.125606+0000","last_became_active":"2023-07-29T13:53:19.560480+0000","last_became_peered":"2023-07-29T13:53:19.560480+0000","last_unstale":"2023-07-29T14:05:33.125606+0000","last_undegraded":"2023-07-29T14:05:33.125606+0000","last_fullsized":"2023-07-29T14:05:33.125606+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:01:49.170231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":910,"reported_epoch":999,"state":"active+clean","last_fresh":"2023-07-29T14:05:33.157987+0000","last_change":"2023-07-29T13:53:19.617225+0000","last_active":"2023-07-29T14:05:33.157987+0000","last_peered":"2023-07-29T14:05:33.157987+0000","last_clean":"2023-07-29T14:05:33.157987+0000","last_became_active":"2023-07-29T13:53:19.611477+0000","last_became_peered":"2023-07-29T13:53:19.611477+0000","last_unstale":"2023-07-29T14:05:33.157987+0000","last_undegraded":"2023-07-29T14:05:33.157987+0000","last_fullsized":"2023-07-29T14:05:33.157987+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:52:56.727655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":948,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.411190+0000","last_change":"2023-07-29T13:56:55.182600+0000","last_active":"2023-07-29T14:05:34.411190+0000","last_peered":"2023-07-29T14:05:34.411190+0000","last_clean":"2023-07-29T14:05:34.411190+0000","last_became_active":"2023-07-29T13:56:55.173228+0000","last_became_peered":"2023-07-29T13:56:55.173228+0000","last_unstale":"2023-07-29T14:05:34.411190+0000","last_undegraded":"2023-07-29T14:05:34.411190+0000","last_fullsized":"2023-07-29T14:05:34.411190+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:57:08.005958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":935,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.452075+0000","last_change":"2023-07-29T13:56:55.243737+0000","last_active":"2023-07-29T14:05:34.452075+0000","last_peered":"2023-07-29T14:05:34.452075+0000","last_clean":"2023-07-29T14:05:34.452075+0000","last_became_active":"2023-07-29T13:56:55.238036+0000","last_became_peered":"2023-07-29T13:56:55.238036+0000","last_unstale":"2023-07-29T14:05:34.452075+0000","last_undegraded":"2023-07-29T14:05:34.452075+0000","last_fullsized":"2023-07-29T14:05:34.452075+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:04:27.494041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":931,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.745324+0000","last_change":"2023-07-29T13:29:33.527250+0000","last_active":"2023-07-29T14:05:36.745324+0000","last_peered":"2023-07-29T14:05:36.745324+0000","last_clean":"2023-07-29T14:05:36.745324+0000","last_became_active":"2023-07-29T13:29:33.508113+0000","last_became_peered":"2023-07-29T13:29:33.508113+0000","last_unstale":"2023-07-29T14:05:36.745324+0000","last_undegraded":"2023-07-29T14:05:36.745324+0000","last_fullsized":"2023-07-29T14:05:36.745324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:52:25.906655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":937,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.193360+0000","last_change":"2023-07-29T13:29:32.261353+0000","last_active":"2023-07-29T14:05:34.193360+0000","last_peered":"2023-07-29T14:05:34.193360+0000","last_clean":"2023-07-29T14:05:34.193360+0000","last_became_active":"2023-07-29T13:29:32.123788+0000","last_became_peered":"2023-07-29T13:29:32.123788+0000","last_unstale":"2023-07-29T14:05:34.193360+0000","last_undegraded":"2023-07-29T14:05:34.193360+0000","last_fullsized":"2023-07-29T14:05:34.193360+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:25:55.926424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":934,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.429328+0000","last_change":"2023-07-29T13:56:55.597264+0000","last_active":"2023-07-29T14:05:34.429328+0000","last_peered":"2023-07-29T14:05:34.429328+0000","last_clean":"2023-07-29T14:05:34.429328+0000","last_became_active":"2023-07-29T13:56:55.591631+0000","last_became_peered":"2023-07-29T13:56:55.591631+0000","last_unstale":"2023-07-29T14:05:34.429328+0000","last_undegraded":"2023-07-29T14:05:34.429328+0000","last_fullsized":"2023-07-29T14:05:34.429328+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:55:19.929908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1003,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.046757+0000","last_change":"2023-07-29T13:56:55.572081+0000","last_active":"2023-07-29T14:05:37.046757+0000","last_peered":"2023-07-29T14:05:37.046757+0000","last_clean":"2023-07-29T14:05:37.046757+0000","last_became_active":"2023-07-29T13:56:55.564155+0000","last_became_peered":"2023-07-29T13:56:55.564155+0000","last_unstale":"2023-07-29T14:05:37.046757+0000","last_undegraded":"2023-07-29T14:05:37.046757+0000","last_fullsized":"2023-07-29T14:05:37.046757+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:56:05.370398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":971,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.554873+0000","last_change":"2023-07-29T13:53:19.441193+0000","last_active":"2023-07-29T14:05:35.554873+0000","last_peered":"2023-07-29T14:05:35.554873+0000","last_clean":"2023-07-29T14:05:35.554873+0000","last_became_active":"2023-07-29T13:53:19.433950+0000","last_became_peered":"2023-07-29T13:53:19.433950+0000","last_unstale":"2023-07-29T14:05:35.554873+0000","last_undegraded":"2023-07-29T14:05:35.554873+0000","last_fullsized":"2023-07-29T14:05:35.554873+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:38:39.350563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":950,"reported_epoch":1002,"state":"peering","last_fresh":"2023-07-29T14:05:36.299801+0000","last_change":"2023-07-29T14:05:36.303408+0000","last_active":"2023-07-29T14:05:36.299801+0000","last_peered":"2023-07-29T14:05:35.502600+0000","last_clean":"2023-07-29T14:04:06.286440+0000","last_became_active":"2023-07-29T14:05:29.629683+0000","last_became_peered":"2023-07-29T14:05:29.629683+0000","last_unstale":"2023-07-29T14:05:36.299801+0000","last_undegraded":"2023-07-29T14:05:36.299801+0000","last_fullsized":"2023-07-29T14:05:36.299801+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:52:32.253454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":62,"acting":62,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":17,"seq":73014444472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60040,"kb_used_data":6848,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":7012352,"data_stored":3836586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":70,"apply_latency_ms":70,"commit_latency_ns":70000000,"apply_latency_ns":70000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":15.343999999999999,"5min":16.634,"15min":170.43700000000001},"min":{"1min":6.3490000000000002,"5min":6.3330000000000002,"15min":4.7590000000000003},"max":{"1min":111.66,"5min":172.44399999999999,"15min":12751.485000000001},"last":8.1539999999999999},{"interface":"front","average":{"1min":16.254000000000001,"5min":16.774000000000001,"15min":173.22},"min":{"1min":6.79,"5min":5.4500000000000002,"15min":5.4500000000000002},"max":{"1min":129.28399999999999,"5min":173.48699999999999,"15min":12747.76},"last":15.557}]},{"osd":1,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":18.536000000000001,"5min":1543.261,"15min":525.65800000000002},"min":{"1min":7.0010000000000003,"5min":6.5149999999999997,"15min":6.5149999999999997},"max":{"1min":99.557000000000002,"5min":29038.089,"15min":29038.089},"last":58.942},{"interface":"front","average":{"1min":22.518000000000001,"5min":1548.174,"15min":530.21500000000003},"min":{"1min":6.9089999999999998,"5min":6.4749999999999996,"15min":6.2249999999999996},"max":{"1min":102.327,"5min":29039.032999999999,"15min":29039.032999999999},"last":53.643999999999998}]},{"osd":2,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":23.297000000000001,"5min":21.439,"15min":20.452000000000002},"min":{"1min":7.992,"5min":7.3200000000000003,"15min":7.3200000000000003},"max":{"1min":104.566,"5min":249.84800000000001,"15min":784.66200000000003},"last":21.434000000000001},{"interface":"front","average":{"1min":21.670000000000002,"5min":21.084,"15min":19.474},"min":{"1min":8.6280000000000001,"5min":8.3919999999999995,"15min":7.8049999999999997},"max":{"1min":99.706000000000003,"5min":202.64500000000001,"15min":368.62200000000001},"last":25.123999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.91,"5min":25.608000000000001,"15min":478.83800000000002},"min":{"1min":7.0910000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"max":{"1min":101.714,"5min":312.73599999999999,"15min":28381.707999999999},"last":22.844000000000001},{"interface":"front","average":{"1min":20.298999999999999,"5min":27.803999999999998,"15min":474.06299999999999},"min":{"1min":9.3219999999999992,"5min":9.3219999999999992,"15min":8.7520000000000007},"max":{"1min":97.727999999999994,"5min":242.63,"15min":28126.012999999999},"last":25.978000000000002}]},{"osd":5,"last update":"Sat Jul 29 14:05:01 2023","interfaces":[{"interface":"back","average":{"1min":17.222999999999999,"5min":20.141999999999999,"15min":32.993000000000002},"min":{"1min":10.452,"5min":8.4540000000000006,"15min":7.3689999999999998},"max":{"1min":79.867000000000004,"5min":85.578999999999994,"15min":488.72300000000001},"last":58.226999999999997},{"interface":"front","average":{"1min":14.161,"5min":19.236999999999998,"15min":21.838000000000001},"min":{"1min":8.1379999999999999,"5min":8.1379999999999999,"15min":7.3840000000000003},"max":{"1min":52.622,"5min":106.634,"15min":128.791},"last":51.494}]},{"osd":6,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":24.904,"5min":20.061,"15min":22.629999999999999},"min":{"1min":9.1470000000000002,"5min":8.0129999999999999,"15min":7.7779999999999996},"max":{"1min":106.33199999999999,"5min":124.819,"15min":787.18499999999995},"last":54.243000000000002},{"interface":"front","average":{"1min":25.198,"5min":20.042000000000002,"15min":22.454999999999998},"min":{"1min":9.8010000000000002,"5min":8.5790000000000006,"15min":8.3170000000000002},"max":{"1min":112.328,"5min":112.328,"15min":646.87599999999998},"last":35.186999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":27.215,"5min":22.292000000000002,"15min":48.816000000000003},"min":{"1min":11.487,"5min":9.2620000000000005,"15min":9.2620000000000005},"max":{"1min":149.43199999999999,"5min":149.43199999999999,"15min":648.54200000000003},"last":36.865000000000002},{"interface":"front","average":{"1min":23.321000000000002,"5min":20.568999999999999,"15min":38.241999999999997},"min":{"1min":10.385,"5min":9.2300000000000004,"15min":8.7309999999999999},"max":{"1min":75.954999999999998,"5min":75.954999999999998,"15min":364.86500000000001},"last":30.257999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444484,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64112,"kb_used_data":6824,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94307728,"statfs":{"total":96636764160,"available":96571113472,"internally_reserved":0,"allocated":6987776,"data_stored":3804439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":13.247999999999999,"15min":130.82599999999999},"min":{"1min":5.3789999999999996,"5min":5.0209999999999999,"15min":4.476},"max":{"1min":29.634,"5min":93.590999999999994,"15min":10009.493},"last":12.676},{"interface":"front","average":{"1min":13.936999999999999,"5min":13.526,"15min":131.76599999999999},"min":{"1min":7.1299999999999999,"5min":6.2069999999999999,"15min":5.617},"max":{"1min":37.527999999999999,"5min":99.156999999999996,"15min":9995.5869999999995},"last":10.468}]},{"osd":1,"last update":"Sat Jul 29 14:04:47 2023","interfaces":[{"interface":"back","average":{"1min":11.255000000000001,"5min":1413.5239999999999,"15min":481.31099999999998},"min":{"1min":6.0899999999999999,"5min":6.0899999999999999,"15min":6.0899999999999999},"max":{"1min":17.850999999999999,"5min":29343.018,"15min":29343.018},"last":101.836},{"interface":"front","average":{"1min":11.234999999999999,"5min":1411.289,"15min":478.77199999999999},"min":{"1min":7.0209999999999999,"5min":6.5880000000000001,"15min":5.8860000000000001},"max":{"1min":18.347000000000001,"5min":29342.263999999999,"15min":29342.263999999999},"last":137.94}]},{"osd":3,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":17.623000000000001,"5min":20.777000000000001,"15min":365.04399999999998},"min":{"1min":6.7809999999999997,"5min":6.7809999999999997,"15min":6.7809999999999997},"max":{"1min":86.926000000000002,"5min":331.92399999999998,"15min":29770.313999999998},"last":11.025},{"interface":"front","average":{"1min":21.684999999999999,"5min":22.039999999999999,"15min":358.54500000000002},"min":{"1min":9.3070000000000004,"5min":8.0540000000000003,"15min":6.4749999999999996},"max":{"1min":89.156000000000006,"5min":332.49900000000002,"15min":29586.387999999999},"last":7.351}]},{"osd":4,"last update":"Sat Jul 29 14:04:53 2023","interfaces":[{"interface":"back","average":{"1min":12.471,"5min":819.34000000000003,"15min":870.13199999999995},"min":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"max":{"1min":52.584000000000003,"5min":25685.629000000001,"15min":32071.659},"last":6.5810000000000004},{"interface":"front","average":{"1min":15.125999999999999,"5min":814.09699999999998,"15min":870.976},"min":{"1min":8.8330000000000002,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":35.020000000000003,"5min":25610.863000000001,"15min":32126.150000000001},"last":8.0410000000000004}]},{"osd":5,"last update":"Sat Jul 29 14:05:06 2023","interfaces":[{"interface":"back","average":{"1min":14.398,"5min":20.428999999999998,"15min":38.106999999999999},"min":{"1min":8.7579999999999991,"5min":8.4269999999999996,"15min":8.4269999999999996},"max":{"1min":18.667000000000002,"5min":193.45400000000001,"15min":428.04000000000002},"last":11.632999999999999},{"interface":"front","average":{"1min":12.019,"5min":20.446999999999999,"15min":41.822000000000003},"min":{"1min":6.891,"5min":6.891,"15min":6.891},"max":{"1min":19.634,"5min":174.292,"15min":430.15499999999997},"last":10.163}]},{"osd":6,"last update":"Sat Jul 29 14:04:38 2023","interfaces":[{"interface":"back","average":{"1min":14.581,"5min":23.431000000000001,"15min":19.853000000000002},"min":{"1min":8.5359999999999996,"5min":8.5359999999999996,"15min":8.5359999999999996},"max":{"1min":24.651,"5min":318.45299999999997,"15min":318.45299999999997},"last":12.380000000000001},{"interface":"front","average":{"1min":15.903,"5min":21.638000000000002,"15min":20.547999999999998},"min":{"1min":11.417999999999999,"5min":9.7789999999999999,"15min":7.5300000000000002},"max":{"1min":22.896999999999998,"5min":349.34899999999999,"15min":349.34899999999999},"last":9.2230000000000008}]},{"osd":7,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":15.279999999999999,"5min":20.536000000000001,"15min":21.027000000000001},"min":{"1min":10.663,"5min":8.5079999999999991,"15min":7.5890000000000004},"max":{"1min":20.757999999999999,"5min":291.80900000000003,"15min":291.80900000000003},"last":25.786000000000001},{"interface":"front","average":{"1min":15.644,"5min":21.513000000000002,"15min":22.518000000000001},"min":{"1min":9.6419999999999995,"5min":9.1940000000000008,"15min":9.1940000000000008},"max":{"1min":23.408000000000001,"5min":291.44200000000001,"15min":291.44200000000001},"last":82.373999999999995}]}]},{"osd":1,"up_from":447,"seq":1919850381742,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60408,"kb_used_data":7216,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311432,"statfs":{"total":96636764160,"available":96574906368,"internally_reserved":0,"allocated":7389184,"data_stored":4202279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":20.931000000000001,"5min":20.015999999999998,"15min":112.807},"min":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.3680000000000003},"max":{"1min":116.512,"5min":364.47800000000001,"15min":12525.57},"last":10.975},{"interface":"front","average":{"1min":18.943000000000001,"5min":20.664000000000001,"15min":112.874},"min":{"1min":6.0970000000000004,"5min":6.0970000000000004,"15min":4.6820000000000004},"max":{"1min":100.402,"5min":372.00999999999999,"15min":12523.974},"last":8.8849999999999998}]},{"osd":2,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.987,"5min":26.809000000000001,"15min":19.399999999999999},"min":{"1min":7.819,"5min":6.5110000000000001,"15min":5.5919999999999996},"max":{"1min":69.497,"5min":360.88099999999997,"15min":360.88099999999997},"last":8.4369999999999994},{"interface":"front","average":{"1min":19.43,"5min":28.155999999999999,"15min":20.445},"min":{"1min":7.093,"5min":7.093,"15min":6.2729999999999997},"max":{"1min":93.792000000000002,"5min":368.51400000000001,"15min":368.51400000000001},"last":7.2009999999999996}]},{"osd":3,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":26.696000000000002,"5min":30.559000000000001,"15min":531.69799999999998},"min":{"1min":9.1590000000000007,"5min":6.7140000000000004,"15min":6.7140000000000004},"max":{"1min":216.16300000000001,"5min":435.92200000000003,"15min":34504.942000000003},"last":18.053000000000001},{"interface":"front","average":{"1min":31.332000000000001,"5min":41.100999999999999,"15min":539.97299999999996},"min":{"1min":10.423,"5min":10.423,"15min":5.3639999999999999},"max":{"1min":238.33699999999999,"5min":700.10500000000002,"15min":34620.970999999998},"last":29.898}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.427,"5min":1170.991,"15min":980.98599999999999},"min":{"1min":8.4130000000000003,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":50.289000000000001,"5min":25247.098999999998,"15min":32258.529999999999},"last":11.036},{"interface":"front","average":{"1min":16.536000000000001,"5min":1165.587,"15min":967.39999999999998},"min":{"1min":11.340999999999999,"5min":8.9169999999999998,"15min":6.5090000000000003},"max":{"1min":29.376999999999999,"5min":25174.008999999998,"15min":31933.992999999999},"last":9.3900000000000006}]},{"osd":5,"last update":"Sat Jul 29 14:05:13 2023","interfaces":[{"interface":"back","average":{"1min":15.249000000000001,"5min":33.526000000000003,"15min":43.616},"min":{"1min":10.416,"5min":10.294,"15min":10.294},"max":{"1min":23.649999999999999,"5min":433.79300000000001,"15min":433.79300000000001},"last":81.477999999999994},{"interface":"front","average":{"1min":15.952,"5min":36.515000000000001,"15min":46.286000000000001},"min":{"1min":11.467000000000001,"5min":9.6929999999999996,"15min":8.7260000000000009},"max":{"1min":22.300999999999998,"5min":419.22800000000001,"15min":419.22800000000001},"last":88.980999999999995}]},{"osd":6,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":16.931999999999999,"5min":21.992000000000001,"15min":19.459},"min":{"1min":6.6079999999999997,"5min":6.327,"15min":6.327},"max":{"1min":154.92400000000001,"5min":357.20499999999998,"15min":357.20499999999998},"last":15.124000000000001},{"interface":"front","average":{"1min":20.251999999999999,"5min":27.460999999999999,"15min":23.411000000000001},"min":{"1min":10.877000000000001,"5min":9.5079999999999991,"15min":9.5079999999999991},"max":{"1min":155.994,"5min":359.68099999999998,"15min":359.68099999999998},"last":14.048}]},{"osd":7,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.683,"5min":35.137999999999998,"15min":23.725999999999999},"min":{"1min":10.130000000000001,"5min":8.5199999999999996,"15min":7.5919999999999996},"max":{"1min":92.168999999999997,"5min":563.29700000000003,"15min":563.29700000000003},"last":12.036},{"interface":"front","average":{"1min":17.949000000000002,"5min":30.442,"15min":21.349},"min":{"1min":8.2240000000000002,"5min":8.2240000000000002,"15min":7.2919999999999998},"max":{"1min":113.15900000000001,"5min":352.65199999999999,"15min":352.65199999999999},"last":9.9580000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444467,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59964,"kb_used_data":6772,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":6934528,"data_stored":3812685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22296,"apply_latency_ms":22296,"commit_latency_ns":22296000000,"apply_latency_ns":22296000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jul 29 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":13.532999999999999,"5min":1013.628,"15min":347.988},"min":{"1min":6.0149999999999997,"5min":5.9249999999999998,"15min":4.6269999999999998},"max":{"1min":29.141999999999999,"5min":27534,"15min":27534},"last":11.698},{"interface":"front","average":{"1min":9.7669999999999995,"5min":1008.927,"15min":345.74400000000003},"min":{"1min":6.2649999999999997,"5min":5.6479999999999997,"15min":5.0739999999999998},"max":{"1min":23.675000000000001,"5min":27535.008999999998,"15min":27535.008999999998},"last":6.5510000000000002}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.170999999999999,"5min":22.285,"15min":18.638000000000002},"min":{"1min":11.680999999999999,"5min":8.1069999999999993,"15min":6.6159999999999997},"max":{"1min":364.62799999999999,"5min":364.62799999999999,"15min":364.62799999999999},"last":364.62799999999999},{"interface":"front","average":{"1min":45.890000000000001,"5min":26.417999999999999,"15min":21.443000000000001},"min":{"1min":14.172000000000001,"5min":10.023999999999999,"15min":8.8780000000000001},"max":{"1min":370.34300000000002,"5min":370.34300000000002,"15min":370.34300000000002},"last":370.34300000000002}]},{"osd":3,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":66.832999999999998,"5min":25.178000000000001,"15min":355.89100000000002},"min":{"1min":6.7610000000000001,"5min":6.4409999999999998,"15min":5.891},"max":{"1min":444.59500000000003,"5min":444.59500000000003,"15min":30170.040000000001},"last":444.59500000000003},{"interface":"front","average":{"1min":65.754000000000005,"5min":33.606000000000002,"15min":363.07400000000001},"min":{"1min":10.266999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"max":{"1min":386.06099999999998,"5min":386.06099999999998,"15min":30171.194},"last":386.06099999999998}]},{"osd":4,"last update":"Sat Jul 29 14:03:48 2023","interfaces":[{"interface":"back","average":{"1min":13.739000000000001,"5min":22.238,"15min":580.12199999999996},"min":{"1min":8.734,"5min":6.383,"15min":6.0979999999999999},"max":{"1min":29.972000000000001,"5min":248.834,"15min":29892.669999999998},"last":14.708},{"interface":"front","average":{"1min":11.521000000000001,"5min":24.349,"15min":582.06399999999996},"min":{"1min":6.7220000000000004,"5min":5.9400000000000004,"15min":5.9400000000000004},"max":{"1min":25.157,"5min":309.649,"15min":29877.289000000001},"last":10.628}]},{"osd":5,"last update":"Sat Jul 29 14:03:02 2023","interfaces":[{"interface":"back","average":{"1min":19.699999999999999,"5min":27.545000000000002,"15min":59.249000000000002},"min":{"1min":9.5700000000000003,"5min":9.5700000000000003,"15min":9.5700000000000003},"max":{"1min":47.115000000000002,"5min":138.58600000000001,"15min":508.02100000000002},"last":17.608000000000001},{"interface":"front","average":{"1min":13.037000000000001,"5min":19.096,"15min":32.008000000000003},"min":{"1min":7.9720000000000004,"5min":7.1210000000000004,"15min":7.1210000000000004},"max":{"1min":47.228999999999999,"5min":152.809,"15min":171.863},"last":15.032}]},{"osd":6,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":52.365000000000002,"5min":30.853999999999999,"15min":25.227},"min":{"1min":16.876000000000001,"5min":10.409000000000001,"15min":9.9350000000000005},"max":{"1min":375.36500000000001,"5min":375.36500000000001,"15min":375.36500000000001},"last":375.36500000000001},{"interface":"front","average":{"1min":55.634,"5min":32.579999999999998,"15min":27.417999999999999},"min":{"1min":18.984999999999999,"5min":11.564,"15min":11.564},"max":{"1min":380.31599999999997,"5min":380.31599999999997,"15min":380.31599999999997},"last":380.31599999999997}]},{"osd":7,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":57.960999999999999,"5min":28.638000000000002,"15min":20.780000000000001},"min":{"1min":17.748999999999999,"5min":8.4939999999999998,"15min":7.9349999999999996},"max":{"1min":385.01900000000001,"5min":385.01900000000001,"15min":385.01900000000001},"last":385.01900000000001},{"interface":"front","average":{"1min":54.853999999999999,"5min":29.765999999999998,"15min":25.545000000000002},"min":{"1min":10.992000000000001,"5min":8.1229999999999993,"15min":8.1229999999999993},"max":{"1min":446.69099999999997,"5min":446.69099999999997,"15min":446.69099999999997},"last":446.69099999999997}]}]},{"osd":4,"up_from":16,"seq":68719477161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60104,"kb_used_data":6912,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311736,"statfs":{"total":96636764160,"available":96575217664,"internally_reserved":0,"allocated":7077888,"data_stored":3907142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":13.432,"5min":21.457000000000001,"15min":87.253},"min":{"1min":6.4029999999999996,"5min":4.6230000000000002,"15min":4.6230000000000002},"max":{"1min":48.048999999999999,"5min":495.86900000000003,"15min":10627.831},"last":11.238},{"interface":"front","average":{"1min":18.010000000000002,"5min":25.013000000000002,"15min":90.852000000000004},"min":{"1min":7.2400000000000002,"5min":6.0990000000000002,"15min":5.6660000000000004},"max":{"1min":124.64100000000001,"5min":446.84100000000001,"15min":10653.877},"last":7.7329999999999997}]},{"osd":1,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":11.276999999999999,"5min":1441.567,"15min":495.98200000000003},"min":{"1min":7.2640000000000002,"5min":7.2640000000000002,"15min":5.9909999999999997},"max":{"1min":21.369,"5min":26872.857,"15min":26872.857},"last":8.4139999999999997},{"interface":"front","average":{"1min":11.013,"5min":1440.0119999999999,"15min":493.40499999999997},"min":{"1min":7.4269999999999996,"5min":7.3090000000000002,"15min":5.6909999999999998},"max":{"1min":17.440999999999999,"5min":26874.048999999999,"15min":26874.048999999999},"last":10.928000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":12.445,"5min":24.050999999999998,"15min":23.050000000000001},"min":{"1min":8.9000000000000004,"5min":7.3019999999999996,"15min":7.3019999999999996},"max":{"1min":15.904999999999999,"5min":453.40100000000001,"15min":597.11800000000005},"last":17.305},{"interface":"front","average":{"1min":15.029999999999999,"5min":27.077999999999999,"15min":26.128},"min":{"1min":12.429,"5min":11.631,"15min":8.0559999999999992},"max":{"1min":19.111000000000001,"5min":453.96199999999999,"15min":705.04499999999996},"last":57.628}]},{"osd":3,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":14.789999999999999,"5min":27.626999999999999,"15min":590.51199999999994},"min":{"1min":9.7219999999999995,"5min":7.6799999999999997,"15min":5.9740000000000002},"max":{"1min":23.942,"5min":448.65100000000001,"15min":30725.612000000001},"last":13.273999999999999},{"interface":"front","average":{"1min":12.497,"5min":20.047999999999998,"15min":582.06899999999996},"min":{"1min":7.7690000000000001,"5min":6.3840000000000003,"15min":6.3840000000000003},"max":{"1min":19.815000000000001,"5min":336.62599999999998,"15min":30407.659},"last":15.597}]},{"osd":5,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":16.190999999999999,"5min":30.579000000000001,"15min":32.869},"min":{"1min":14.1,"5min":7.1440000000000001,"15min":7.1440000000000001},"max":{"1min":18.738,"5min":450.81700000000001,"15min":450.81700000000001},"last":14.425000000000001},{"interface":"front","average":{"1min":10.430999999999999,"5min":19.959,"15min":22.803999999999998},"min":{"1min":6.2779999999999996,"5min":6.2779999999999996,"15min":6.2779999999999996},"max":{"1min":20.254999999999999,"5min":340.04000000000002,"15min":340.04000000000002},"last":7.9279999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":13.007,"5min":22.312000000000001,"15min":22.866},"min":{"1min":8.1120000000000001,"5min":7.1689999999999996,"15min":6.1520000000000001},"max":{"1min":22.010000000000002,"5min":339.47199999999998,"15min":696.952},"last":10.199999999999999},{"interface":"front","average":{"1min":14.724,"5min":23.917999999999999,"15min":23.209},"min":{"1min":9.0289999999999999,"5min":6.3449999999999998,"15min":6.3449999999999998},"max":{"1min":20.968,"5min":498.26999999999998,"15min":598.86900000000003},"last":16.390999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":12.805,"5min":24.097000000000001,"15min":24.802},"min":{"1min":8.7430000000000003,"5min":8.3399999999999999,"15min":7.7930000000000001},"max":{"1min":18.638000000000002,"5min":340.61900000000003,"15min":699.31799999999998},"last":13.837999999999999},{"interface":"front","average":{"1min":17.483000000000001,"5min":31.318999999999999,"15min":33.040999999999997},"min":{"1min":13.861000000000001,"5min":11.067,"15min":11.067},"max":{"1min":19.817,"5min":455.339,"15min":703.68200000000002},"last":11.691000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82276,"kb_used_data":6940,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94289564,"statfs":{"total":96636764160,"available":96552513536,"internally_reserved":0,"allocated":7106560,"data_stored":3935814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":12.891999999999999,"5min":16.984000000000002,"15min":62.423000000000002},"min":{"1min":6.9390000000000001,"5min":5.2350000000000003,"15min":5.2350000000000003},"max":{"1min":29.821000000000002,"5min":224.93899999999999,"15min":9502.5869999999995},"last":41.209000000000003},{"interface":"front","average":{"1min":15.925000000000001,"5min":19.867000000000001,"15min":65.921999999999997},"min":{"1min":8.2050000000000001,"5min":6.1580000000000004,"15min":6.1580000000000004},"max":{"1min":31.303999999999998,"5min":130.48400000000001,"15min":9511.3719999999994},"last":64.659000000000006}]},{"osd":1,"last update":"Sat Jul 29 14:04:49 2023","interfaces":[{"interface":"back","average":{"1min":17.140999999999998,"5min":970.20899999999995,"15min":335.46300000000002},"min":{"1min":7.5839999999999996,"5min":7.3369999999999997,"15min":4.8529999999999998},"max":{"1min":73.679000000000002,"5min":26903.921999999999,"15min":26903.921999999999},"last":8.5850000000000009},{"interface":"front","average":{"1min":12.878,"5min":963.24699999999996,"15min":332.12599999999998},"min":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.3330000000000002},"max":{"1min":69.432000000000002,"5min":26893.022000000001,"15min":26893.022000000001},"last":5.3620000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":14.891999999999999,"5min":21.295999999999999,"15min":19.027000000000001},"min":{"1min":10.173,"5min":7.8609999999999998,"15min":6.6299999999999999},"max":{"1min":26.302,"5min":191.73099999999999,"15min":191.73099999999999},"last":26.302},{"interface":"front","average":{"1min":11.848000000000001,"5min":16.003,"15min":15.422000000000001},"min":{"1min":7.4690000000000003,"5min":5.5499999999999998,"15min":5.5499999999999998},"max":{"1min":18,"5min":189.40199999999999,"15min":189.40199999999999},"last":11.739000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":10.653,"5min":15.525,"15min":521.48800000000006},"min":{"1min":7.2619999999999996,"5min":6.335,"15min":6.335},"max":{"1min":23.292000000000002,"5min":187.852,"15min":30710.619999999999},"last":37.712000000000003},{"interface":"front","average":{"1min":15.202999999999999,"5min":17.899000000000001,"15min":521.02300000000002},"min":{"1min":9.907,"5min":7.7009999999999996,"15min":7.6180000000000003},"max":{"1min":40.335000000000001,"5min":101.88200000000001,"15min":30591.776999999998},"last":38.539999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:56 2023","interfaces":[{"interface":"back","average":{"1min":16.030000000000001,"5min":808.61300000000006,"15min":1019.547},"min":{"1min":8.0950000000000006,"5min":6.718,"15min":6.1070000000000002},"max":{"1min":73.055999999999997,"5min":25620.438999999998,"15min":32899.582000000002},"last":8.2870000000000008},{"interface":"front","average":{"1min":16.128,"5min":828.35799999999995,"15min":1009.853},"min":{"1min":10.254,"5min":6.2089999999999996,"15min":6.2089999999999996},"max":{"1min":40.786999999999999,"5min":25979.712,"15min":32485.278999999999},"last":9.7769999999999992}]},{"osd":5,"last update":"Sat Jul 29 14:05:04 2023","interfaces":[{"interface":"back","average":{"1min":17.489999999999998,"5min":21.376999999999999,"15min":24.847000000000001},"min":{"1min":11.52,"5min":9.4949999999999992,"15min":8.9290000000000003},"max":{"1min":34.677,"5min":75.25,"15min":189.423},"last":94.355000000000004},{"interface":"front","average":{"1min":14.324,"5min":18.073,"15min":21.690000000000001},"min":{"1min":8.6969999999999992,"5min":7.3330000000000002,"15min":7.3330000000000002},"max":{"1min":24.77,"5min":80.085999999999999,"15min":188.518},"last":91.700999999999993}]},{"osd":7,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":24.321000000000002,"5min":19.489999999999998,"15min":19.814},"min":{"1min":14.164,"5min":7.1059999999999999,"15min":7.1059999999999999},"max":{"1min":74.337000000000003,"5min":74.337000000000003,"15min":200.55099999999999},"last":13.702999999999999},{"interface":"front","average":{"1min":18.581,"5min":20.780000000000001,"15min":22.265000000000001},"min":{"1min":9.6880000000000006,"5min":8.0790000000000006,"15min":8.0790000000000006},"max":{"1min":47.313000000000002,"5min":89.125,"15min":222.65199999999999},"last":12.701000000000001}]}]},{"osd":5,"up_from":704,"seq":3023656976772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78444,"kb_used_data":7204,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94293396,"statfs":{"total":96636764160,"available":96556437504,"internally_reserved":0,"allocated":7376896,"data_stored":4202582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":78,"apply_latency_ms":78,"commit_latency_ns":78000000,"apply_latency_ns":78000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:04:00 2023","interfaces":[{"interface":"back","average":{"1min":23.48,"5min":22.623000000000001,"15min":41.000999999999998},"min":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":4.742},"max":{"1min":150.63,"5min":262.56099999999998,"15min":299.87799999999999},"last":9.6039999999999992},{"interface":"front","average":{"1min":24.837,"5min":22.727,"15min":52.545999999999999},"min":{"1min":5.8259999999999996,"5min":4.6539999999999999,"15min":4.6539999999999999},"max":{"1min":111.14100000000001,"5min":253.80000000000001,"15min":370.00999999999999},"last":7.3899999999999997}]},{"osd":1,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":13.867000000000001,"5min":25.277000000000001,"15min":33.274999999999999},"min":{"1min":6.5419999999999998,"5min":5.3780000000000001,"15min":5.3780000000000001},"max":{"1min":91.590000000000003,"5min":362.58800000000002,"15min":362.58800000000002},"last":7.0529999999999999},{"interface":"front","average":{"1min":15.465999999999999,"5min":34.079000000000001,"15min":56.872999999999998},"min":{"1min":7.5650000000000004,"5min":6.8680000000000003,"15min":6.8680000000000003},"max":{"1min":85.760000000000005,"5min":356.69900000000001,"15min":363.50999999999999},"last":9.3510000000000009}]},{"osd":2,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.152000000000001,"5min":26.193000000000001,"15min":51.703000000000003},"min":{"1min":8.1880000000000006,"5min":7.032,"15min":7.032},"max":{"1min":151.804,"5min":312.755,"15min":384.024},"last":6.0259999999999998},{"interface":"front","average":{"1min":16.754000000000001,"5min":20.221,"15min":25.251000000000001},"min":{"1min":7.782,"5min":7.2759999999999998,"15min":7.2709999999999999},"max":{"1min":102.09099999999999,"5min":228.55600000000001,"15min":228.55600000000001},"last":11.063000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":28.802,"15min":55.957999999999998},"min":{"1min":10.734,"5min":7.4100000000000001,"15min":7.4100000000000001},"max":{"1min":131.36600000000001,"5min":312.12200000000001,"15min":381.23200000000003},"last":12.308},{"interface":"front","average":{"1min":18.568999999999999,"5min":28.875,"15min":66.894999999999996},"min":{"1min":7.944,"5min":7.944,"15min":7.944},"max":{"1min":118.456,"5min":265.92700000000002,"15min":382.779},"last":24.484999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":19.385999999999999,"5min":1108.5630000000001,"15min":412.19600000000003},"min":{"1min":10.5,"5min":7.343,"15min":7.3070000000000004},"max":{"1min":103.768,"5min":23572.741999999998,"15min":23572.741999999998},"last":8.423},{"interface":"front","average":{"1min":20.039000000000001,"5min":1113.885,"15min":428.73500000000001},"min":{"1min":8.7260000000000009,"5min":6.8929999999999998,"15min":6.8929999999999998},"max":{"1min":137.57400000000001,"5min":23610.142,"15min":23610.142},"last":10.224}]},{"osd":6,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":17.074000000000002,"5min":25.704999999999998,"15min":51.972000000000001},"min":{"1min":9.7360000000000007,"5min":8.1709999999999994,"15min":8.1709999999999994},"max":{"1min":41.347000000000001,"5min":263.983,"15min":450.17500000000001},"last":25.594000000000001},{"interface":"front","average":{"1min":18.436,"5min":26.684999999999999,"15min":57.575000000000003},"min":{"1min":11.185,"5min":8.1969999999999992,"15min":8.1969999999999992},"max":{"1min":63.707999999999998,"5min":221.99199999999999,"15min":417.303},"last":14.269}]},{"osd":7,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":21.550999999999998,"5min":28.064,"15min":47.530999999999999},"min":{"1min":12.340999999999999,"5min":11.222,"15min":10.602},"max":{"1min":104.718,"5min":284.48000000000002,"15min":341.30500000000001},"last":12.962999999999999},{"interface":"front","average":{"1min":18.460999999999999,"5min":27.806000000000001,"15min":53.993000000000002},"min":{"1min":12.089,"5min":11.042999999999999,"15min":11.042999999999999},"max":{"1min":46.353999999999999,"5min":191.33099999999999,"15min":329.86500000000001},"last":28.317}]}]},{"osd":7,"up_from":650,"seq":2791728742799,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60024,"kb_used_data":6832,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311816,"statfs":{"total":96636764160,"available":96575299584,"internally_reserved":0,"allocated":6995968,"data_stored":3838376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:45 2023","interfaces":[{"interface":"back","average":{"1min":11.94,"5min":13.635,"15min":23.015000000000001},"min":{"1min":5.5890000000000004,"5min":5.4539999999999997,"15min":5.4539999999999997},"max":{"1min":28.5,"5min":75.968000000000004,"15min":293.94499999999999},"last":11.353},{"interface":"front","average":{"1min":14.968999999999999,"5min":18.658999999999999,"15min":28.620000000000001},"min":{"1min":6.8179999999999996,"5min":6.8179999999999996,"15min":6.8179999999999996},"max":{"1min":29.143999999999998,"5min":116.952,"15min":358.267},"last":12.553000000000001}]},{"osd":1,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":19.048999999999999,"5min":1108.855,"15min":389.67099999999999},"min":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":4.6660000000000004},"max":{"1min":184.25800000000001,"5min":27546.963,"15min":27546.963},"last":13.448},{"interface":"front","average":{"1min":22.969999999999999,"5min":1111.8620000000001,"15min":385.10300000000001},"min":{"1min":8.6110000000000007,"5min":5.5010000000000003,"15min":5.5010000000000003},"max":{"1min":150.75700000000001,"5min":27546.280999999999,"15min":27546.280999999999},"last":9.6600000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.088000000000001,"5min":19.381,"15min":28.02},"min":{"1min":10.154999999999999,"5min":7.9100000000000001,"15min":7.9100000000000001},"max":{"1min":103.09,"5min":103.09,"15min":461.83800000000002},"last":14.393000000000001},{"interface":"front","average":{"1min":21.068000000000001,"5min":15.704000000000001,"15min":23.861999999999998},"min":{"1min":7.1269999999999998,"5min":6.6239999999999997,"15min":6.6239999999999997},"max":{"1min":181.97900000000001,"5min":181.97900000000001,"15min":385.40800000000002},"last":8.4250000000000007}]},{"osd":3,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":25.065999999999999,"5min":21.029,"15min":1531.4280000000001},"min":{"1min":7.976,"5min":7.6660000000000004,"15min":7.6660000000000004},"max":{"1min":242.197,"5min":242.197,"15min":34550.838000000003},"last":15.186999999999999},{"interface":"front","average":{"1min":31.428999999999998,"5min":27.873000000000001,"15min":1543.2929999999999},"min":{"1min":12.592000000000001,"5min":6.9550000000000001,"15min":6.9550000000000001},"max":{"1min":240.46899999999999,"5min":243.06399999999999,"15min":34556.381000000001},"last":16.431000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":20.402999999999999,"5min":1074.817,"15min":900.84500000000003},"min":{"1min":6.7220000000000004,"5min":6.3719999999999999,"15min":6.3719999999999999},"max":{"1min":103.931,"5min":25146.41,"15min":27891.68},"last":18.895},{"interface":"front","average":{"1min":29.045999999999999,"5min":1099.6300000000001,"15min":918.40200000000004},"min":{"1min":9.7360000000000007,"5min":8.1630000000000003,"15min":7.2699999999999996},"max":{"1min":189.47900000000001,"5min":25434.151999999998,"15min":27883.419999999998},"last":11.879}]},{"osd":5,"last update":"Sat Jul 29 14:05:08 2023","interfaces":[{"interface":"back","average":{"1min":17.529,"5min":28.314,"15min":72.040999999999997},"min":{"1min":14.367000000000001,"5min":11.215,"15min":10.683},"max":{"1min":22.012,"5min":230.16800000000001,"15min":1265.123},"last":23.506},{"interface":"front","average":{"1min":12.343999999999999,"5min":25.687999999999999,"15min":76.659000000000006},"min":{"1min":9.609,"5min":7.71,"15min":7.5540000000000003},"max":{"1min":21.030999999999999,"5min":255.23699999999999,"15min":1234.9300000000001},"last":20.945}]},{"osd":6,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":21.149999999999999,"15min":25.884},"min":{"1min":10.222,"5min":9.9740000000000002,"15min":7.8689999999999998},"max":{"1min":78.843000000000004,"5min":95.558999999999997,"15min":243.66499999999999},"last":17.23},{"interface":"front","average":{"1min":24.378,"5min":25.236999999999998,"15min":32.558},"min":{"1min":12.111000000000001,"5min":10.542,"15min":10.542},"max":{"1min":92.266999999999996,"5min":148.334,"15min":454.68799999999999},"last":18.457999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-29T14:05:39.538 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-29T14:05:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.650+0000 7f36e5a99640 10 monclient: get_monmap_and_config 2023-07-29T14:05:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.650+0000 7f36e5a99640 10 monclient: build_initial_monmap 2023-07-29T14:05:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.650+0000 7f36e5a99640 10 monclient: monmap: 2023-07-29T14:05:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:39.652110+0000 2023-07-29T14:05:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:39.652110+0000 2023-07-29T14:05:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:39.655 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.651+0000 7f36e5a99640 1 Processor -- start 2023-07-29T14:05:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.651+0000 7f36e5a99640 1 -- start start 2023-07-29T14:05:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.651+0000 7f36e5a99640 10 monclient: init 2023-07-29T14:05:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 10 monclient: _add_conns ranks=[0,4,3,1,2] 2023-07-29T14:05:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f36e00b3ae0 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36e013e3a0 0x7f36e0140c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 10 monclient(hunting): picked mon.noname-c con 0x7f36e013e3a0 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36e00a8910 0x7f36e0141150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 10 monclient(hunting): picked mon.noname-a con 0x7f36e00a8910 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e00b3eb0 con 0x7f36e00b3ae0 2023-07-29T14:05:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e00b0f10 con 0x7f36e00a8910 2023-07-29T14:05:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e00a9950 con 0x7f36e013e3a0 2023-07-29T14:05:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:39.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.652+0000 7f36e5a99640 10 monclient(hunting): authenticate will time out at 3661.785681s 2023-07-29T14:05:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.653+0000 7f36d6ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36e013e3a0 0x7f36e0140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.653+0000 7f36d6ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36e013e3a0 0x7f36e0140c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:57428/0 (socket says 172.21.15.40:57428) 2023-07-29T14:05:39.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.653+0000 7f36d6ffd640 1 -- 172.21.15.40:0/2844700963 learned_addr learned my addr 172.21.15.40:0/2844700963 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:39.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.653+0000 7f36d6ffd640 10 monclient(hunting): get_auth_request con 0x7f36e013e3a0 auth_method 0 2023-07-29T14:05:39.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.653+0000 7f36d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:39.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.653+0000 7f36d6ffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:39.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.653+0000 7f36d6ffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:39.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.654+0000 7f36d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:39.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.654+0000 7f36d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.654+0000 7f36d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.654+0000 7f36dffff640 1 --2- 172.21.15.40:0/2844700963 >> v2:172.21.15.40:3300/0 conn(0x7f36e00a8910 0x7f36e0141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.654+0000 7f36dffff640 10 monclient(hunting): get_auth_request con 0x7f36e00a8910 auth_method 0 2023-07-29T14:05:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.654+0000 7f36dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.654+0000 7f36dffff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.654+0000 7f36dffff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.655+0000 7f36dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.655+0000 7f36dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.655+0000 7f36dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:39.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.657+0000 7f36d6ffd640 10 monclient(hunting): handle_auth_done global_id 20308 payload 274 2023-07-29T14:05:39.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.657+0000 7f36d6ffd640 1 -- 172.21.15.40:0/2844700963 >> v2:172.21.15.40:3300/0 conn(0x7f36e00a8910 msgr2=0x7f36e0141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:39.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.657+0000 7f36d6ffd640 1 --2- 172.21.15.40:0/2844700963 >> v2:172.21.15.40:3300/0 conn(0x7f36e00a8910 0x7f36e0141150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.657+0000 7f36d6ffd640 1 -- 172.21.15.40:0/2844700963 >> v1:172.21.15.40:6789/0 conn(0x7f36e00b3ae0 legacy=0x7f36e013dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:39.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.657+0000 7f36d6ffd640 10 monclient: _finish_hunting 0 2023-07-29T14:05:39.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.657+0000 7f36d6ffd640 1 monclient: found mon.noname-c 2023-07-29T14:05:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.657+0000 7f36d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.657+0000 7f36d6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T14:05:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.657+0000 7f36d6ffd640 1 -- 172.21.15.40:0/2844700963 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e0141690 con 0x7f36e013e3a0 2023-07-29T14:05:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.659+0000 7f36d6ffd640 1 --2- 172.21.15.40:0/2844700963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36e013e3a0 0x7f36e0140c10 secure :-1 s=READY pgs=4775 cs=0 l=1 rev1=1 crypto rx=0x7f36e0070a70 tx=0x7f36c802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1db79d898cdb54c server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 1 -- 172.21.15.40:0/2844700963 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8031870 con 0x7f36e013e3a0 2023-07-29T14:05:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 10 monclient: dump: 2023-07-29T14:05:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:39.669 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 10 monclient: _finish_auth 0 2023-07-29T14:05:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 1 -- 172.21.15.40:0/2844700963 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c80319d0 con 0x7f36e013e3a0 2023-07-29T14:05:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36e5a99640 5 monclient: authenticate success, global_id 20308 2023-07-29T14:05:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36e5a99640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T14:05:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36e5a99640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36e5a99640 4 set_mon_vals no callback set 2023-07-29T14:05:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 1 -- 172.21.15.40:0/2844700963 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8031d60 con 0x7f36e013e3a0 2023-07-29T14:05:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T14:05:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36deffd640 10 monclient: dump: 2023-07-29T14:05:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:39.673 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36e5a99640 10 monclient: shutdown 2023-07-29T14:05:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36e5a99640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36e5a99640 1 -- 172.21.15.40:0/2844700963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36e013e3a0 msgr2=0x7f36e0140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.664+0000 7f36e5a99640 1 --2- 172.21.15.40:0/2844700963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36e013e3a0 0x7f36e0140c10 secure :-1 s=READY pgs=4775 cs=0 l=1 rev1=1 crypto rx=0x7f36e0070a70 tx=0x7f36c802b910 comp rx=0 tx=0).stop 2023-07-29T14:05:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.665+0000 7f36e5a99640 1 -- 172.21.15.40:0/2844700963 shutdown_connections 2023-07-29T14:05:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.665+0000 7f36e5a99640 1 --2- 172.21.15.40:0/2844700963 >> v2:172.21.15.40:3300/0 conn(0x7f36e00a8910 0x7f36e0141150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.665+0000 7f36e5a99640 1 --2- 172.21.15.40:0/2844700963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36e013e3a0 0x7f36e0140c10 unknown :-1 s=CLOSED pgs=4775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.665+0000 7f36e5a99640 1 -- 172.21.15.40:0/2844700963 >> 172.21.15.40:0/2844700963 conn(0x7f36e0070fc0 msgr2=0x7f36e00713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.665+0000 7f36e5a99640 1 -- 172.21.15.40:0/2844700963 shutdown_connections 2023-07-29T14:05:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.665+0000 7f36e5a99640 1 -- 172.21.15.40:0/2844700963 wait complete. 2023-07-29T14:05:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.665+0000 7f36e5a99640 10 monclient: build_initial_monmap 2023-07-29T14:05:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.665+0000 7f36e5a99640 10 monclient: monmap: 2023-07-29T14:05:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:39.676 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 librados: starting msgr at 2023-07-29T14:05:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 librados: starting objecter 2023-07-29T14:05:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 Processor -- start 2023-07-29T14:05:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 -- start start 2023-07-29T14:05:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 librados: setting wanted keys 2023-07-29T14:05:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 librados: calling monclient init 2023-07-29T14:05:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 10 monclient: init 2023-07-29T14:05:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T14:05:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36e00a8910 0x7f36e00b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 10 monclient(hunting): picked mon.noname-c con 0x7f36e00a8910 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36e00b3ae0 0x7f36e01d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 10 monclient(hunting): picked mon.noname-b con 0x7f36e00b3ae0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e013e3a0 0x7f36e01d5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 10 monclient(hunting): picked mon.noname-a con 0x7f36e013e3a0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e01441c0 con 0x7f36e013e3a0 2023-07-29T14:05:39.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:39.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e00b75b0 con 0x7f36e00b3ae0 2023-07-29T14:05:39.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:39.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e00b7720 con 0x7f36e00a8910 2023-07-29T14:05:39.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:39.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.666+0000 7f36e5a99640 10 monclient(hunting): authenticate will time out at 3661.800028s 2023-07-29T14:05:39.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.667+0000 7f36df7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36e00a8910 0x7f36e00b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.667+0000 7f36d6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36e00b3ae0 0x7f36e01d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.667+0000 7f36df7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36e00a8910 0x7f36e00b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:37220/0 (socket says 172.21.15.40:37220) 2023-07-29T14:05:39.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.667+0000 7f36df7fe640 1 -- 172.21.15.40:0/2973143333 learned_addr learned my addr 172.21.15.40:0/2973143333 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:39.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.667+0000 7f36df7fe640 10 monclient(hunting): get_auth_request con 0x7f36e00a8910 auth_method 0 2023-07-29T14:05:39.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.667+0000 7f36df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:39.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.667+0000 7f36df7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:39.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.667+0000 7f36df7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:39.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.668+0000 7f36d6ffd640 10 monclient(hunting): get_auth_request con 0x7f36e00b3ae0 auth_method 0 2023-07-29T14:05:39.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.668+0000 7f36d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:39.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.668+0000 7f36d6ffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:39.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.668+0000 7f36d6ffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:39.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.668+0000 7f36df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:39.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.668+0000 7f36df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:39.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.668+0000 7f36df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:39.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.668+0000 7f36d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.668+0000 7f36d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.668+0000 7f36d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.669+0000 7f36dffff640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e013e3a0 0x7f36e01d5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.670+0000 7f36dffff640 10 monclient(hunting): get_auth_request con 0x7f36e013e3a0 auth_method 0 2023-07-29T14:05:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.670+0000 7f36dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.670+0000 7f36dffff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.670+0000 7f36dffff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.670+0000 7f36dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.670+0000 7f36dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.670+0000 7f36dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.671+0000 7f36df7fe640 10 monclient(hunting): handle_auth_done global_id 16340 payload 824 2023-07-29T14:05:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.671+0000 7f36df7fe640 1 -- 172.21.15.40:0/2973143333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36e00b3ae0 msgr2=0x7f36e01d5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.671+0000 7f36df7fe640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36e00b3ae0 0x7f36e01d5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.671+0000 7f36df7fe640 1 -- 172.21.15.40:0/2973143333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e013e3a0 msgr2=0x7f36e01d5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.671+0000 7f36df7fe640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e013e3a0 0x7f36e01d5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.671+0000 7f36df7fe640 10 monclient: _finish_hunting 0 2023-07-29T14:05:39.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.671+0000 7f36df7fe640 1 monclient: found mon.noname-c 2023-07-29T14:05:39.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.671+0000 7f36df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:39.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.671+0000 7f36df7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:39.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.671+0000 7f36df7fe640 1 -- 172.21.15.40:0/2973143333 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c80094d0 con 0x7f36e00a8910 2023-07-29T14:05:39.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.674+0000 7f36df7fe640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36e00a8910 0x7f36e00b6c30 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 crypto rx=0x7f36d000c500 tx=0x7f36d000c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:39.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.674+0000 7f36dcff9640 10 client.?.objecter ms_handle_connect 0x7f36e00a8910 2023-07-29T14:05:39.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.674+0000 7f36dcff9640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:39.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 1 -- 172.21.15.40:0/2973143333 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d00052d0 con 0x7f36e00a8910 2023-07-29T14:05:39.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:39.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:39.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 10 monclient: dump: 2023-07-29T14:05:39.689 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:39.689 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:39.689 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.689 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.689 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:39.689 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.690 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:39.690 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:39.690 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:39.690 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 10 monclient: _finish_auth 0 2023-07-29T14:05:39.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 10 monclient: _check_auth_tickets 2023-07-29T14:05:39.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:39.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 1 -- 172.21.15.40:0/2973143333 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d0005430 con 0x7f36e00a8910 2023-07-29T14:05:39.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36e5a99640 5 monclient: authenticate success, global_id 16340 2023-07-29T14:05:39.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36e5a99640 10 monclient: _renew_subs 2023-07-29T14:05:39.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36e5a99640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:39.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36e5a99640 1 -- 172.21.15.40:0/2973143333 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e0143810 con 0x7f36e00a8910 2023-07-29T14:05:39.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:39.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 1 -- 172.21.15.40:0/2973143333 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0012850 con 0x7f36e00a8910 2023-07-29T14:05:39.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:39.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:39.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36dcff9640 10 monclient: dump: 2023-07-29T14:05:39.692 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:39.692 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:39.692 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.692 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:39.692 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:39.693 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:39.693 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:39.693 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:39.693 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:39.693 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:39.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36de7fc640 4 set_mon_vals no callback set 2023-07-29T14:05:39.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36e5a99640 10 client.16340.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:39.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36e5a99640 10 monclient: _renew_subs 2023-07-29T14:05:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36e5a99640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36e5a99640 1 -- 172.21.15.40:0/2973143333 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e0144830 con 0x7f36e00a8910 2023-07-29T14:05:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.680+0000 7f36e5a99640 1 librados: init done 2023-07-29T14:05:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.681+0000 7f36e5a99640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.681+0000 7f36e5a99640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.681+0000 7f36e5a99640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.681+0000 7f36e5a99640 1 -- 172.21.15.40:0/2973143333 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e006bbb0 con 0x7f36e00a8910 2023-07-29T14:05:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.943+0000 7f36dcff9640 1 -- 172.21.15.40:0/2973143333 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f36d00129b0 con 0x7f36e00a8910 2023-07-29T14:05:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.944+0000 7f36dcff9640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f36c006bfb0 0x7f36c006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.944+0000 7f36d6ffd640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f36c006bfb0 0x7f36c006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:39.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.944+0000 7f36d6ffd640 10 monclient: get_auth_request con 0x7f36c006bfb0 auth_method 0 2023-07-29T14:05:39.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.945+0000 7f36d6ffd640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f36c006bfb0 0x7f36c006e460 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f36c8036000 tx=0x7f36c803c040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:39.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.945+0000 7f36dcff9640 10 client.16340.objecter ms_handle_connect 0x7f36c006bfb0 2023-07-29T14:05:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.956+0000 7f36dcff9640 1 -- 172.21.15.40:0/2973143333 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f36d005bfd0 con 0x7f36e00a8910 2023-07-29T14:05:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.956+0000 7f36dcff9640 10 client.16340.objecter ms_dispatch 0x7f36e0000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.956+0000 7f36dcff9640 3 client.16340.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.956+0000 7f36dcff9640 3 client.16340.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.956+0000 7f36dcff9640 20 client.16340.objecter dump_active .. 0 homeless 2023-07-29T14:05:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.956+0000 7f36dcff9640 1 -- 172.21.15.40:0/2973143333 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f36d0054400 con 0x7f36e00a8910 2023-07-29T14:05:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.956+0000 7f36dcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:39.956+0000 7f36dcff9640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:40.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.140+0000 7f36e5a99640 1 -- 172.21.15.40:0/2973143333 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36e00b2d40 con 0x7f36c006bfb0 2023-07-29T14:05:40.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.145+0000 7f36dcff9640 1 -- 172.21.15.40:0/2973143333 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152067 (secure 0 0 0) 0x7f36c8031670 con 0x7f36c006bfb0 2023-07-29T14:05:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T14:05:40.149 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-29T14:05:40.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.147+0000 7f36e5a99640 10 librados: watch_flush enter 2023-07-29T14:05:40.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.147+0000 7f36e5a99640 10 librados: watch_flush exit 2023-07-29T14:05:40.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.147+0000 7f36e5a99640 20 client.16340.objecter shutdown clearing up homeless session... 2023-07-29T14:05:40.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.147+0000 7f36e5a99640 10 client.16340.objecter successfully canceled tick 2023-07-29T14:05:40.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 -- 172.21.15.40:0/2973143333 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f36c006bfb0 msgr2=0x7f36c006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f36c006bfb0 0x7f36c006e460 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f36c8036000 tx=0x7f36c803c040 comp rx=0 tx=0).stop 2023-07-29T14:05:40.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 10 monclient: shutdown 2023-07-29T14:05:40.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:40.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 -- 172.21.15.40:0/2973143333 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36e00a8910 msgr2=0x7f36e00b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36e00a8910 0x7f36e00b6c30 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 crypto rx=0x7f36d000c500 tx=0x7f36d000c9c0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 -- 172.21.15.40:0/2973143333 shutdown_connections 2023-07-29T14:05:40.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f36c006bfb0 0x7f36c006e460 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36e00b3ae0 0x7f36e01d5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e013e3a0 0x7f36e01d5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 --2- 172.21.15.40:0/2973143333 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36e00a8910 0x7f36e00b6c30 unknown :-1 s=CLOSED pgs=3216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 -- 172.21.15.40:0/2973143333 >> 172.21.15.40:0/2973143333 conn(0x7f36e0070fc0 msgr2=0x7f36e013c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:40.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 -- 172.21.15.40:0/2973143333 shutdown_connections 2023-07-29T14:05:40.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 -- 172.21.15.40:0/2973143333 wait complete. 2023-07-29T14:05:40.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.148+0000 7f36e5a99640 1 librados: shutdown 2023-07-29T14:05:40.159 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":2131,"stamp":"2023-07-29T14:05:39.629046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":81,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":525432,"kb_used_data":54888,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754449288,"statfs":{"total":773094113280,"available":772556070912,"internally_reserved":0,"allocated":56205312,"data_stored":31064796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":452,"apply_latency_ms":452,"commit_latency_ns":452000000,"apply_latency_ns":452000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4000299,"num_objects":-44,"num_object_clones":-1,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44,"num_whiteouts":0,"num_read":-31,"num_read_kb":-11,"num_write":-71,"num_write_kb":-3969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.483953"},"pg_stats":[{"pgid":"7.1b","version":"0'0","reported_seq":968,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.762797+0000","last_change":"2023-07-29T13:56:55.662556+0000","last_active":"2023-07-29T14:05:36.762797+0000","last_peered":"2023-07-29T14:05:36.762797+0000","last_clean":"2023-07-29T14:05:36.762797+0000","last_became_active":"2023-07-29T13:56:55.656930+0000","last_became_peered":"2023-07-29T13:56:55.656930+0000","last_unstale":"2023-07-29T14:05:36.762797+0000","last_undegraded":"2023-07-29T14:05:36.762797+0000","last_fullsized":"2023-07-29T14:05:36.762797+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:52:51.321547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":977,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.016088+0000","last_change":"2023-07-29T13:41:38.328763+0000","last_active":"2023-07-29T14:05:37.016088+0000","last_peered":"2023-07-29T14:05:37.016088+0000","last_clean":"2023-07-29T14:05:37.016088+0000","last_became_active":"2023-07-29T13:41:38.322679+0000","last_became_peered":"2023-07-29T13:41:38.322679+0000","last_unstale":"2023-07-29T14:05:37.016088+0000","last_undegraded":"2023-07-29T14:05:37.016088+0000","last_fullsized":"2023-07-29T14:05:37.016088+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:40:55.886733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":964,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.997533+0000","last_change":"2023-07-29T13:29:52.105383+0000","last_active":"2023-07-29T14:05:36.997533+0000","last_peered":"2023-07-29T14:05:36.997533+0000","last_clean":"2023-07-29T14:05:36.997533+0000","last_became_active":"2023-07-29T13:29:52.099163+0000","last_became_peered":"2023-07-29T13:29:52.099163+0000","last_unstale":"2023-07-29T14:05:36.997533+0000","last_undegraded":"2023-07-29T14:05:36.997533+0000","last_fullsized":"2023-07-29T14:05:36.997533+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:03:36.535099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":930,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.709810+0000","last_change":"2023-07-29T13:29:41.816664+0000","last_active":"2023-07-29T14:05:36.709810+0000","last_peered":"2023-07-29T14:05:36.709810+0000","last_clean":"2023-07-29T14:05:36.709810+0000","last_became_active":"2023-07-29T13:29:41.782506+0000","last_became_peered":"2023-07-29T13:29:41.782506+0000","last_unstale":"2023-07-29T14:05:36.709810+0000","last_undegraded":"2023-07-29T14:05:36.709810+0000","last_fullsized":"2023-07-29T14:05:36.709810+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:20:57.566653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":947,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.066403+0000","last_change":"2023-07-29T14:05:38.066403+0000","last_active":"2023-07-29T14:05:38.066403+0000","last_peered":"2023-07-29T14:05:38.066403+0000","last_clean":"2023-07-29T14:05:38.066403+0000","last_became_active":"2023-07-29T14:05:38.060277+0000","last_became_peered":"2023-07-29T14:05:38.060277+0000","last_unstale":"2023-07-29T14:05:38.066403+0000","last_undegraded":"2023-07-29T14:05:38.066403+0000","last_fullsized":"2023-07-29T14:05:38.066403+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:50:25.119138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":913,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.286005+0000","last_change":"2023-07-29T13:53:19.606247+0000","last_active":"2023-07-29T14:05:36.286005+0000","last_peered":"2023-07-29T14:05:36.286005+0000","last_clean":"2023-07-29T14:05:36.286005+0000","last_became_active":"2023-07-29T13:53:19.600386+0000","last_became_peered":"2023-07-29T13:53:19.600386+0000","last_unstale":"2023-07-29T14:05:36.286005+0000","last_undegraded":"2023-07-29T14:05:36.286005+0000","last_fullsized":"2023-07-29T14:05:36.286005+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:46:03.055229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":934,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.436676+0000","last_change":"2023-07-29T13:56:55.406251+0000","last_active":"2023-07-29T14:05:34.436676+0000","last_peered":"2023-07-29T14:05:34.436676+0000","last_clean":"2023-07-29T14:05:34.436676+0000","last_became_active":"2023-07-29T13:56:55.400591+0000","last_became_peered":"2023-07-29T13:56:55.400591+0000","last_unstale":"2023-07-29T14:05:34.436676+0000","last_undegraded":"2023-07-29T14:05:34.436676+0000","last_fullsized":"2023-07-29T14:05:34.436676+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:40:48.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":954,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.369590+0000","last_change":"2023-07-29T13:29:31.089676+0000","last_active":"2023-07-29T14:05:36.369590+0000","last_peered":"2023-07-29T14:05:36.369590+0000","last_clean":"2023-07-29T14:05:36.369590+0000","last_became_active":"2023-07-29T13:29:31.072042+0000","last_became_peered":"2023-07-29T13:29:31.072042+0000","last_unstale":"2023-07-29T14:05:36.369590+0000","last_undegraded":"2023-07-29T14:05:36.369590+0000","last_fullsized":"2023-07-29T14:05:36.369590+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:28:28.141003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":912,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.243324+0000","last_change":"2023-07-29T13:53:19.677067+0000","last_active":"2023-07-29T14:05:36.243324+0000","last_peered":"2023-07-29T14:05:36.243324+0000","last_clean":"2023-07-29T14:05:36.243324+0000","last_became_active":"2023-07-29T13:53:19.671215+0000","last_became_peered":"2023-07-29T13:53:19.671215+0000","last_unstale":"2023-07-29T14:05:36.243324+0000","last_undegraded":"2023-07-29T14:05:36.243324+0000","last_fullsized":"2023-07-29T14:05:36.243324+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:51:18.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":985,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.445062+0000","last_change":"2023-07-29T13:56:55.517147+0000","last_active":"2023-07-29T14:05:35.445062+0000","last_peered":"2023-07-29T14:05:35.445062+0000","last_clean":"2023-07-29T14:05:35.445062+0000","last_became_active":"2023-07-29T13:56:55.511490+0000","last_became_peered":"2023-07-29T13:56:55.511490+0000","last_unstale":"2023-07-29T14:05:35.445062+0000","last_undegraded":"2023-07-29T14:05:35.445062+0000","last_fullsized":"2023-07-29T14:05:35.445062+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:26.433868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":962,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.507347+0000","last_change":"2023-07-29T14:05:37.515837+0000","last_active":"2023-07-29T14:05:36.327038+0000","last_peered":"2023-07-29T14:05:35.596454+0000","last_clean":"2023-07-29T14:04:22.472067+0000","last_became_active":"2023-07-29T14:05:29.733569+0000","last_became_peered":"2023-07-29T14:05:29.733569+0000","last_unstale":"2023-07-29T14:05:37.507347+0000","last_undegraded":"2023-07-29T14:05:37.507347+0000","last_fullsized":"2023-07-29T14:05:37.507347+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:03:53.476296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":913,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.470943+0000","last_change":"2023-07-29T13:53:19.644608+0000","last_active":"2023-07-29T14:05:37.470943+0000","last_peered":"2023-07-29T14:05:37.470943+0000","last_clean":"2023-07-29T14:05:37.470943+0000","last_became_active":"2023-07-29T13:53:19.637473+0000","last_became_peered":"2023-07-29T13:53:19.637473+0000","last_unstale":"2023-07-29T14:05:37.470943+0000","last_undegraded":"2023-07-29T14:05:37.470943+0000","last_fullsized":"2023-07-29T14:05:37.470943+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:53:02.384899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":932,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.746967+0000","last_change":"2023-07-29T13:29:33.530600+0000","last_active":"2023-07-29T14:05:36.746967+0000","last_peered":"2023-07-29T14:05:36.746967+0000","last_clean":"2023-07-29T14:05:36.746967+0000","last_became_active":"2023-07-29T13:29:33.515255+0000","last_became_peered":"2023-07-29T13:29:33.515255+0000","last_unstale":"2023-07-29T14:05:36.746967+0000","last_undegraded":"2023-07-29T14:05:36.746967+0000","last_fullsized":"2023-07-29T14:05:36.746967+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:21:56.794070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":979,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.433490+0000","last_change":"2023-07-29T14:05:37.441418+0000","last_active":"2023-07-29T14:05:36.969928+0000","last_peered":"2023-07-29T14:05:35.332022+0000","last_clean":"2023-07-29T14:05:12.448301+0000","last_became_active":"2023-07-29T14:05:29.733673+0000","last_became_peered":"2023-07-29T14:05:29.733673+0000","last_unstale":"2023-07-29T14:05:37.433490+0000","last_undegraded":"2023-07-29T14:05:37.433490+0000","last_fullsized":"2023-07-29T14:05:37.433490+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:38:04.875711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":936,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.418624+0000","last_change":"2023-07-29T13:56:55.146603+0000","last_active":"2023-07-29T14:05:34.418624+0000","last_peered":"2023-07-29T14:05:34.418624+0000","last_clean":"2023-07-29T14:05:34.418624+0000","last_became_active":"2023-07-29T13:56:55.139167+0000","last_became_peered":"2023-07-29T13:56:55.139167+0000","last_unstale":"2023-07-29T14:05:34.418624+0000","last_undegraded":"2023-07-29T14:05:34.418624+0000","last_fullsized":"2023-07-29T14:05:34.418624+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:56:23.477235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":944,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.533539+0000","last_change":"2023-07-29T14:05:37.544652+0000","last_active":"2023-07-29T14:05:36.720848+0000","last_peered":"2023-07-29T14:05:35.604442+0000","last_clean":"2023-07-29T14:04:06.556968+0000","last_became_active":"2023-07-29T14:05:29.660537+0000","last_became_peered":"2023-07-29T14:05:29.660537+0000","last_unstale":"2023-07-29T14:05:37.533539+0000","last_undegraded":"2023-07-29T14:05:37.533539+0000","last_fullsized":"2023-07-29T14:05:37.533539+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:42:16.790121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":987,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.436816+0000","last_change":"2023-07-29T13:56:55.073408+0000","last_active":"2023-07-29T14:05:35.436816+0000","last_peered":"2023-07-29T14:05:35.436816+0000","last_clean":"2023-07-29T14:05:35.436816+0000","last_became_active":"2023-07-29T13:56:55.065724+0000","last_became_peered":"2023-07-29T13:56:55.065724+0000","last_unstale":"2023-07-29T14:05:35.436816+0000","last_undegraded":"2023-07-29T14:05:35.436816+0000","last_fullsized":"2023-07-29T14:05:35.436816+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:42:58.825948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":971,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.050103+0000","last_change":"2023-07-29T13:28:14.434312+0000","last_active":"2023-07-29T14:05:37.050103+0000","last_peered":"2023-07-29T14:05:37.050103+0000","last_clean":"2023-07-29T14:05:37.050103+0000","last_became_active":"2023-07-29T13:28:14.423044+0000","last_became_peered":"2023-07-29T13:28:14.423044+0000","last_unstale":"2023-07-29T14:05:37.050103+0000","last_undegraded":"2023-07-29T14:05:37.050103+0000","last_fullsized":"2023-07-29T14:05:37.050103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:56:27.865715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1","version":"70'7","reported_seq":983,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.310515+0000","last_change":"2023-07-29T13:41:38.528285+0000","last_active":"2023-07-29T14:05:36.310515+0000","last_peered":"2023-07-29T14:05:36.310515+0000","last_clean":"2023-07-29T14:05:36.310515+0000","last_became_active":"2023-07-29T13:41:38.521847+0000","last_became_peered":"2023-07-29T13:41:38.521847+0000","last_unstale":"2023-07-29T14:05:36.310515+0000","last_undegraded":"2023-07-29T14:05:36.310515+0000","last_fullsized":"2023-07-29T14:05:36.310515+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:08:42.836378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":942,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.430995+0000","last_change":"2023-07-29T13:56:55.263959+0000","last_active":"2023-07-29T14:05:34.430995+0000","last_peered":"2023-07-29T14:05:34.430995+0000","last_clean":"2023-07-29T14:05:34.430995+0000","last_became_active":"2023-07-29T13:56:55.258241+0000","last_became_peered":"2023-07-29T13:56:55.258241+0000","last_unstale":"2023-07-29T14:05:34.430995+0000","last_undegraded":"2023-07-29T14:05:34.430995+0000","last_fullsized":"2023-07-29T14:05:34.430995+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:42.517469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":966,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.065287+0000","last_change":"2023-07-29T13:29:52.230014+0000","last_active":"2023-07-29T14:05:37.065287+0000","last_peered":"2023-07-29T14:05:37.065287+0000","last_clean":"2023-07-29T14:05:37.065287+0000","last_became_active":"2023-07-29T13:29:52.224467+0000","last_became_peered":"2023-07-29T13:29:52.224467+0000","last_unstale":"2023-07-29T14:05:37.065287+0000","last_undegraded":"2023-07-29T14:05:37.065287+0000","last_fullsized":"2023-07-29T14:05:37.065287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:51:03.019114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":972,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.014430+0000","last_change":"2023-07-29T13:28:14.652586+0000","last_active":"2023-07-29T14:05:37.014430+0000","last_peered":"2023-07-29T14:05:37.014430+0000","last_clean":"2023-07-29T14:05:37.014430+0000","last_became_active":"2023-07-29T13:28:14.642023+0000","last_became_peered":"2023-07-29T13:28:14.642023+0000","last_unstale":"2023-07-29T14:05:37.014430+0000","last_undegraded":"2023-07-29T14:05:37.014430+0000","last_fullsized":"2023-07-29T14:05:37.014430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:25:23.334956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":933,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.764552+0000","last_change":"2023-07-29T13:29:41.869711+0000","last_active":"2023-07-29T14:05:36.764552+0000","last_peered":"2023-07-29T14:05:36.764552+0000","last_clean":"2023-07-29T14:05:36.764552+0000","last_became_active":"2023-07-29T13:29:41.844314+0000","last_became_peered":"2023-07-29T13:29:41.844314+0000","last_unstale":"2023-07-29T14:05:36.764552+0000","last_undegraded":"2023-07-29T14:05:36.764552+0000","last_fullsized":"2023-07-29T14:05:36.764552+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:41:40.503015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":963,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.209438+0000","last_change":"2023-07-29T13:56:55.159956+0000","last_active":"2023-07-29T14:05:36.209438+0000","last_peered":"2023-07-29T14:05:36.209438+0000","last_clean":"2023-07-29T14:05:36.209438+0000","last_became_active":"2023-07-29T13:56:55.150514+0000","last_became_peered":"2023-07-29T13:56:55.150514+0000","last_unstale":"2023-07-29T14:05:36.209438+0000","last_undegraded":"2023-07-29T14:05:36.209438+0000","last_fullsized":"2023-07-29T14:05:36.209438+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:16:38.329000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1005,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.048452+0000","last_change":"2023-07-29T13:56:55.615681+0000","last_active":"2023-07-29T14:05:37.048452+0000","last_peered":"2023-07-29T14:05:37.048452+0000","last_clean":"2023-07-29T14:05:37.048452+0000","last_became_active":"2023-07-29T13:56:55.609414+0000","last_became_peered":"2023-07-29T13:56:55.609414+0000","last_unstale":"2023-07-29T14:05:37.048452+0000","last_undegraded":"2023-07-29T14:05:37.048452+0000","last_fullsized":"2023-07-29T14:05:37.048452+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:36:30.601005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1012,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.999968+0000","last_change":"2023-07-29T13:56:55.550112+0000","last_active":"2023-07-29T14:05:36.999968+0000","last_peered":"2023-07-29T14:05:36.999968+0000","last_clean":"2023-07-29T14:05:36.999968+0000","last_became_active":"2023-07-29T13:56:55.543861+0000","last_became_peered":"2023-07-29T13:56:55.543861+0000","last_unstale":"2023-07-29T14:05:36.999968+0000","last_undegraded":"2023-07-29T14:05:36.999968+0000","last_fullsized":"2023-07-29T14:05:36.999968+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:57:36.179121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":940,"reported_epoch":1000,"state":"active+undersized","last_fresh":"2023-07-29T14:05:34.441767+0000","last_change":"2023-07-29T14:05:29.759408+0000","last_active":"2023-07-29T14:05:34.441767+0000","last_peered":"2023-07-29T14:05:34.441767+0000","last_clean":"2023-07-29T14:04:06.482879+0000","last_became_active":"2023-07-29T14:05:29.759408+0000","last_became_peered":"2023-07-29T14:05:29.759408+0000","last_unstale":"2023-07-29T14:05:34.441767+0000","last_undegraded":"2023-07-29T14:05:34.441767+0000","last_fullsized":"2023-07-29T14:05:29.709468+0000","mapping_epoch":995,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:11:56.037546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":972,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.010320+0000","last_change":"2023-07-29T13:28:14.713485+0000","last_active":"2023-07-29T14:05:37.010320+0000","last_peered":"2023-07-29T14:05:37.010320+0000","last_clean":"2023-07-29T14:05:37.010320+0000","last_became_active":"2023-07-29T13:28:14.708042+0000","last_became_peered":"2023-07-29T13:28:14.708042+0000","last_unstale":"2023-07-29T14:05:37.010320+0000","last_undegraded":"2023-07-29T14:05:37.010320+0000","last_fullsized":"2023-07-29T14:05:37.010320+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:25:56.525512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":1099,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.438875+0000","last_change":"2023-07-29T13:56:55.164457+0000","last_active":"2023-07-29T14:05:35.438875+0000","last_peered":"2023-07-29T14:05:35.438875+0000","last_clean":"2023-07-29T14:05:35.438875+0000","last_became_active":"2023-07-29T13:56:55.157702+0000","last_became_peered":"2023-07-29T13:56:55.157702+0000","last_unstale":"2023-07-29T14:05:35.438875+0000","last_undegraded":"2023-07-29T14:05:35.438875+0000","last_fullsized":"2023-07-29T14:05:35.438875+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_clean_scrub_stamp":"2023-07-29T13:28:06.434639+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:06:43.352094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":995,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.367084+0000","last_change":"2023-07-29T13:56:55.347022+0000","last_active":"2023-07-29T14:05:36.367084+0000","last_peered":"2023-07-29T14:05:36.367084+0000","last_clean":"2023-07-29T14:05:36.367084+0000","last_became_active":"2023-07-29T13:56:55.341426+0000","last_became_peered":"2023-07-29T13:56:55.341426+0000","last_unstale":"2023-07-29T14:05:36.367084+0000","last_undegraded":"2023-07-29T14:05:36.367084+0000","last_fullsized":"2023-07-29T14:05:36.367084+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:46:20.874037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":945,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.466789+0000","last_change":"2023-07-29T13:41:38.063359+0000","last_active":"2023-07-29T14:05:35.466789+0000","last_peered":"2023-07-29T14:05:35.466789+0000","last_clean":"2023-07-29T14:05:35.466789+0000","last_became_active":"2023-07-29T13:41:38.057649+0000","last_became_peered":"2023-07-29T13:41:38.057649+0000","last_unstale":"2023-07-29T14:05:35.466789+0000","last_undegraded":"2023-07-29T14:05:35.466789+0000","last_fullsized":"2023-07-29T14:05:35.466789+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:36:46.868780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":925,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.290603+0000","last_change":"2023-07-29T13:53:19.566408+0000","last_active":"2023-07-29T14:05:36.290603+0000","last_peered":"2023-07-29T14:05:36.290603+0000","last_clean":"2023-07-29T14:05:36.290603+0000","last_became_active":"2023-07-29T13:53:19.560480+0000","last_became_peered":"2023-07-29T13:53:19.560480+0000","last_unstale":"2023-07-29T14:05:36.290603+0000","last_undegraded":"2023-07-29T14:05:36.290603+0000","last_fullsized":"2023-07-29T14:05:36.290603+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:01:49.170231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":913,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.314881+0000","last_change":"2023-07-29T13:53:19.617225+0000","last_active":"2023-07-29T14:05:36.314881+0000","last_peered":"2023-07-29T14:05:36.314881+0000","last_clean":"2023-07-29T14:05:36.314881+0000","last_became_active":"2023-07-29T13:53:19.611477+0000","last_became_peered":"2023-07-29T13:53:19.611477+0000","last_unstale":"2023-07-29T14:05:36.314881+0000","last_undegraded":"2023-07-29T14:05:36.314881+0000","last_fullsized":"2023-07-29T14:05:36.314881+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:52:56.727655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":948,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.411190+0000","last_change":"2023-07-29T13:56:55.182600+0000","last_active":"2023-07-29T14:05:34.411190+0000","last_peered":"2023-07-29T14:05:34.411190+0000","last_clean":"2023-07-29T14:05:34.411190+0000","last_became_active":"2023-07-29T13:56:55.173228+0000","last_became_peered":"2023-07-29T13:56:55.173228+0000","last_unstale":"2023-07-29T14:05:34.411190+0000","last_undegraded":"2023-07-29T14:05:34.411190+0000","last_fullsized":"2023-07-29T14:05:34.411190+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:57:08.005958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":935,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.452075+0000","last_change":"2023-07-29T13:56:55.243737+0000","last_active":"2023-07-29T14:05:34.452075+0000","last_peered":"2023-07-29T14:05:34.452075+0000","last_clean":"2023-07-29T14:05:34.452075+0000","last_became_active":"2023-07-29T13:56:55.238036+0000","last_became_peered":"2023-07-29T13:56:55.238036+0000","last_unstale":"2023-07-29T14:05:34.452075+0000","last_undegraded":"2023-07-29T14:05:34.452075+0000","last_fullsized":"2023-07-29T14:05:34.452075+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:04:27.494041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":931,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.745324+0000","last_change":"2023-07-29T13:29:33.527250+0000","last_active":"2023-07-29T14:05:36.745324+0000","last_peered":"2023-07-29T14:05:36.745324+0000","last_clean":"2023-07-29T14:05:36.745324+0000","last_became_active":"2023-07-29T13:29:33.508113+0000","last_became_peered":"2023-07-29T13:29:33.508113+0000","last_unstale":"2023-07-29T14:05:36.745324+0000","last_undegraded":"2023-07-29T14:05:36.745324+0000","last_fullsized":"2023-07-29T14:05:36.745324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:52:25.906655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":937,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.193360+0000","last_change":"2023-07-29T13:29:32.261353+0000","last_active":"2023-07-29T14:05:34.193360+0000","last_peered":"2023-07-29T14:05:34.193360+0000","last_clean":"2023-07-29T14:05:34.193360+0000","last_became_active":"2023-07-29T13:29:32.123788+0000","last_became_peered":"2023-07-29T13:29:32.123788+0000","last_unstale":"2023-07-29T14:05:34.193360+0000","last_undegraded":"2023-07-29T14:05:34.193360+0000","last_fullsized":"2023-07-29T14:05:34.193360+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:25:55.926424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":934,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.429328+0000","last_change":"2023-07-29T13:56:55.597264+0000","last_active":"2023-07-29T14:05:34.429328+0000","last_peered":"2023-07-29T14:05:34.429328+0000","last_clean":"2023-07-29T14:05:34.429328+0000","last_became_active":"2023-07-29T13:56:55.591631+0000","last_became_peered":"2023-07-29T13:56:55.591631+0000","last_unstale":"2023-07-29T14:05:34.429328+0000","last_undegraded":"2023-07-29T14:05:34.429328+0000","last_fullsized":"2023-07-29T14:05:34.429328+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:55:19.929908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1003,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.046757+0000","last_change":"2023-07-29T13:56:55.572081+0000","last_active":"2023-07-29T14:05:37.046757+0000","last_peered":"2023-07-29T14:05:37.046757+0000","last_clean":"2023-07-29T14:05:37.046757+0000","last_became_active":"2023-07-29T13:56:55.564155+0000","last_became_peered":"2023-07-29T13:56:55.564155+0000","last_unstale":"2023-07-29T14:05:37.046757+0000","last_undegraded":"2023-07-29T14:05:37.046757+0000","last_fullsized":"2023-07-29T14:05:37.046757+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:56:05.370398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":971,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.554873+0000","last_change":"2023-07-29T13:53:19.441193+0000","last_active":"2023-07-29T14:05:35.554873+0000","last_peered":"2023-07-29T14:05:35.554873+0000","last_clean":"2023-07-29T14:05:35.554873+0000","last_became_active":"2023-07-29T13:53:19.433950+0000","last_became_peered":"2023-07-29T13:53:19.433950+0000","last_unstale":"2023-07-29T14:05:35.554873+0000","last_undegraded":"2023-07-29T14:05:35.554873+0000","last_fullsized":"2023-07-29T14:05:35.554873+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:38:39.350563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":950,"reported_epoch":1002,"state":"peering","last_fresh":"2023-07-29T14:05:36.299801+0000","last_change":"2023-07-29T14:05:36.303408+0000","last_active":"2023-07-29T14:05:36.299801+0000","last_peered":"2023-07-29T14:05:35.502600+0000","last_clean":"2023-07-29T14:04:06.286440+0000","last_became_active":"2023-07-29T14:05:29.629683+0000","last_became_peered":"2023-07-29T14:05:29.629683+0000","last_unstale":"2023-07-29T14:05:36.299801+0000","last_undegraded":"2023-07-29T14:05:36.299801+0000","last_fullsized":"2023-07-29T14:05:36.299801+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:52:32.253454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":17,"seq":73014444472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60040,"kb_used_data":6848,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":7012352,"data_stored":3836586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":70,"apply_latency_ms":70,"commit_latency_ns":70000000,"apply_latency_ns":70000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":15.343999999999999,"5min":16.634,"15min":170.43700000000001},"min":{"1min":6.3490000000000002,"5min":6.3330000000000002,"15min":4.7590000000000003},"max":{"1min":111.66,"5min":172.44399999999999,"15min":12751.485000000001},"last":8.1539999999999999},{"interface":"front","average":{"1min":16.254000000000001,"5min":16.774000000000001,"15min":173.22},"min":{"1min":6.79,"5min":5.4500000000000002,"15min":5.4500000000000002},"max":{"1min":129.28399999999999,"5min":173.48699999999999,"15min":12747.76},"last":15.557}]},{"osd":1,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":18.536000000000001,"5min":1543.261,"15min":525.65800000000002},"min":{"1min":7.0010000000000003,"5min":6.5149999999999997,"15min":6.5149999999999997},"max":{"1min":99.557000000000002,"5min":29038.089,"15min":29038.089},"last":58.942},{"interface":"front","average":{"1min":22.518000000000001,"5min":1548.174,"15min":530.21500000000003},"min":{"1min":6.9089999999999998,"5min":6.4749999999999996,"15min":6.2249999999999996},"max":{"1min":102.327,"5min":29039.032999999999,"15min":29039.032999999999},"last":53.643999999999998}]},{"osd":2,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":23.297000000000001,"5min":21.439,"15min":20.452000000000002},"min":{"1min":7.992,"5min":7.3200000000000003,"15min":7.3200000000000003},"max":{"1min":104.566,"5min":249.84800000000001,"15min":784.66200000000003},"last":21.434000000000001},{"interface":"front","average":{"1min":21.670000000000002,"5min":21.084,"15min":19.474},"min":{"1min":8.6280000000000001,"5min":8.3919999999999995,"15min":7.8049999999999997},"max":{"1min":99.706000000000003,"5min":202.64500000000001,"15min":368.62200000000001},"last":25.123999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.91,"5min":25.608000000000001,"15min":478.83800000000002},"min":{"1min":7.0910000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"max":{"1min":101.714,"5min":312.73599999999999,"15min":28381.707999999999},"last":22.844000000000001},{"interface":"front","average":{"1min":20.298999999999999,"5min":27.803999999999998,"15min":474.06299999999999},"min":{"1min":9.3219999999999992,"5min":9.3219999999999992,"15min":8.7520000000000007},"max":{"1min":97.727999999999994,"5min":242.63,"15min":28126.012999999999},"last":25.978000000000002}]},{"osd":5,"last update":"Sat Jul 29 14:05:01 2023","interfaces":[{"interface":"back","average":{"1min":17.222999999999999,"5min":20.141999999999999,"15min":32.993000000000002},"min":{"1min":10.452,"5min":8.4540000000000006,"15min":7.3689999999999998},"max":{"1min":79.867000000000004,"5min":85.578999999999994,"15min":488.72300000000001},"last":58.226999999999997},{"interface":"front","average":{"1min":14.161,"5min":19.236999999999998,"15min":21.838000000000001},"min":{"1min":8.1379999999999999,"5min":8.1379999999999999,"15min":7.3840000000000003},"max":{"1min":52.622,"5min":106.634,"15min":128.791},"last":51.494}]},{"osd":6,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":24.904,"5min":20.061,"15min":22.629999999999999},"min":{"1min":9.1470000000000002,"5min":8.0129999999999999,"15min":7.7779999999999996},"max":{"1min":106.33199999999999,"5min":124.819,"15min":787.18499999999995},"last":54.243000000000002},{"interface":"front","average":{"1min":25.198,"5min":20.042000000000002,"15min":22.454999999999998},"min":{"1min":9.8010000000000002,"5min":8.5790000000000006,"15min":8.3170000000000002},"max":{"1min":112.328,"5min":112.328,"15min":646.87599999999998},"last":35.186999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":27.215,"5min":22.292000000000002,"15min":48.816000000000003},"min":{"1min":11.487,"5min":9.2620000000000005,"15min":9.2620000000000005},"max":{"1min":149.43199999999999,"5min":149.43199999999999,"15min":648.54200000000003},"last":36.865000000000002},{"interface":"front","average":{"1min":23.321000000000002,"5min":20.568999999999999,"15min":38.241999999999997},"min":{"1min":10.385,"5min":9.2300000000000004,"15min":8.7309999999999999},"max":{"1min":75.954999999999998,"5min":75.954999999999998,"15min":364.86500000000001},"last":30.257999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444484,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64112,"kb_used_data":6824,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94307728,"statfs":{"total":96636764160,"available":96571113472,"internally_reserved":0,"allocated":6987776,"data_stored":3804439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":13.247999999999999,"15min":130.82599999999999},"min":{"1min":5.3789999999999996,"5min":5.0209999999999999,"15min":4.476},"max":{"1min":29.634,"5min":93.590999999999994,"15min":10009.493},"last":12.676},{"interface":"front","average":{"1min":13.936999999999999,"5min":13.526,"15min":131.76599999999999},"min":{"1min":7.1299999999999999,"5min":6.2069999999999999,"15min":5.617},"max":{"1min":37.527999999999999,"5min":99.156999999999996,"15min":9995.5869999999995},"last":10.468}]},{"osd":1,"last update":"Sat Jul 29 14:04:47 2023","interfaces":[{"interface":"back","average":{"1min":11.255000000000001,"5min":1413.5239999999999,"15min":481.31099999999998},"min":{"1min":6.0899999999999999,"5min":6.0899999999999999,"15min":6.0899999999999999},"max":{"1min":17.850999999999999,"5min":29343.018,"15min":29343.018},"last":101.836},{"interface":"front","average":{"1min":11.234999999999999,"5min":1411.289,"15min":478.77199999999999},"min":{"1min":7.0209999999999999,"5min":6.5880000000000001,"15min":5.8860000000000001},"max":{"1min":18.347000000000001,"5min":29342.263999999999,"15min":29342.263999999999},"last":137.94}]},{"osd":3,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":17.623000000000001,"5min":20.777000000000001,"15min":365.04399999999998},"min":{"1min":6.7809999999999997,"5min":6.7809999999999997,"15min":6.7809999999999997},"max":{"1min":86.926000000000002,"5min":331.92399999999998,"15min":29770.313999999998},"last":11.025},{"interface":"front","average":{"1min":21.684999999999999,"5min":22.039999999999999,"15min":358.54500000000002},"min":{"1min":9.3070000000000004,"5min":8.0540000000000003,"15min":6.4749999999999996},"max":{"1min":89.156000000000006,"5min":332.49900000000002,"15min":29586.387999999999},"last":7.351}]},{"osd":4,"last update":"Sat Jul 29 14:04:53 2023","interfaces":[{"interface":"back","average":{"1min":12.471,"5min":819.34000000000003,"15min":870.13199999999995},"min":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"max":{"1min":52.584000000000003,"5min":25685.629000000001,"15min":32071.659},"last":6.5810000000000004},{"interface":"front","average":{"1min":15.125999999999999,"5min":814.09699999999998,"15min":870.976},"min":{"1min":8.8330000000000002,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":35.020000000000003,"5min":25610.863000000001,"15min":32126.150000000001},"last":8.0410000000000004}]},{"osd":5,"last update":"Sat Jul 29 14:05:06 2023","interfaces":[{"interface":"back","average":{"1min":14.398,"5min":20.428999999999998,"15min":38.106999999999999},"min":{"1min":8.7579999999999991,"5min":8.4269999999999996,"15min":8.4269999999999996},"max":{"1min":18.667000000000002,"5min":193.45400000000001,"15min":428.04000000000002},"last":11.632999999999999},{"interface":"front","average":{"1min":12.019,"5min":20.446999999999999,"15min":41.822000000000003},"min":{"1min":6.891,"5min":6.891,"15min":6.891},"max":{"1min":19.634,"5min":174.292,"15min":430.15499999999997},"last":10.163}]},{"osd":6,"last update":"Sat Jul 29 14:04:38 2023","interfaces":[{"interface":"back","average":{"1min":14.581,"5min":23.431000000000001,"15min":19.853000000000002},"min":{"1min":8.5359999999999996,"5min":8.5359999999999996,"15min":8.5359999999999996},"max":{"1min":24.651,"5min":318.45299999999997,"15min":318.45299999999997},"last":12.380000000000001},{"interface":"front","average":{"1min":15.903,"5min":21.638000000000002,"15min":20.547999999999998},"min":{"1min":11.417999999999999,"5min":9.7789999999999999,"15min":7.5300000000000002},"max":{"1min":22.896999999999998,"5min":349.34899999999999,"15min":349.34899999999999},"last":9.2230000000000008}]},{"osd":7,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":15.279999999999999,"5min":20.536000000000001,"15min":21.027000000000001},"min":{"1min":10.663,"5min":8.5079999999999991,"15min":7.5890000000000004},"max":{"1min":20.757999999999999,"5min":291.80900000000003,"15min":291.80900000000003},"last":25.786000000000001},{"interface":"front","average":{"1min":15.644,"5min":21.513000000000002,"15min":22.518000000000001},"min":{"1min":9.6419999999999995,"5min":9.1940000000000008,"15min":9.1940000000000008},"max":{"1min":23.408000000000001,"5min":291.44200000000001,"15min":291.44200000000001},"last":82.373999999999995}]}]},{"osd":1,"up_from":447,"seq":1919850381742,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60408,"kb_used_data":7216,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311432,"statfs":{"total":96636764160,"available":96574906368,"internally_reserved":0,"allocated":7389184,"data_stored":4202279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":20.931000000000001,"5min":20.015999999999998,"15min":112.807},"min":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.3680000000000003},"max":{"1min":116.512,"5min":364.47800000000001,"15min":12525.57},"last":10.975},{"interface":"front","average":{"1min":18.943000000000001,"5min":20.664000000000001,"15min":112.874},"min":{"1min":6.0970000000000004,"5min":6.0970000000000004,"15min":4.6820000000000004},"max":{"1min":100.402,"5min":372.00999999999999,"15min":12523.974},"last":8.8849999999999998}]},{"osd":2,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.987,"5min":26.809000000000001,"15min":19.399999999999999},"min":{"1min":7.819,"5min":6.5110000000000001,"15min":5.5919999999999996},"max":{"1min":69.497,"5min":360.88099999999997,"15min":360.88099999999997},"last":8.4369999999999994},{"interface":"front","average":{"1min":19.43,"5min":28.155999999999999,"15min":20.445},"min":{"1min":7.093,"5min":7.093,"15min":6.2729999999999997},"max":{"1min":93.792000000000002,"5min":368.51400000000001,"15min":368.51400000000001},"last":7.2009999999999996}]},{"osd":3,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":26.696000000000002,"5min":30.559000000000001,"15min":531.69799999999998},"min":{"1min":9.1590000000000007,"5min":6.7140000000000004,"15min":6.7140000000000004},"max":{"1min":216.16300000000001,"5min":435.92200000000003,"15min":34504.942000000003},"last":18.053000000000001},{"interface":"front","average":{"1min":31.332000000000001,"5min":41.100999999999999,"15min":539.97299999999996},"min":{"1min":10.423,"5min":10.423,"15min":5.3639999999999999},"max":{"1min":238.33699999999999,"5min":700.10500000000002,"15min":34620.970999999998},"last":29.898}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.427,"5min":1170.991,"15min":980.98599999999999},"min":{"1min":8.4130000000000003,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":50.289000000000001,"5min":25247.098999999998,"15min":32258.529999999999},"last":11.036},{"interface":"front","average":{"1min":16.536000000000001,"5min":1165.587,"15min":967.39999999999998},"min":{"1min":11.340999999999999,"5min":8.9169999999999998,"15min":6.5090000000000003},"max":{"1min":29.376999999999999,"5min":25174.008999999998,"15min":31933.992999999999},"last":9.3900000000000006}]},{"osd":5,"last update":"Sat Jul 29 14:05:13 2023","interfaces":[{"interface":"back","average":{"1min":15.249000000000001,"5min":33.526000000000003,"15min":43.616},"min":{"1min":10.416,"5min":10.294,"15min":10.294},"max":{"1min":23.649999999999999,"5min":433.79300000000001,"15min":433.79300000000001},"last":81.477999999999994},{"interface":"front","average":{"1min":15.952,"5min":36.515000000000001,"15min":46.286000000000001},"min":{"1min":11.467000000000001,"5min":9.6929999999999996,"15min":8.7260000000000009},"max":{"1min":22.300999999999998,"5min":419.22800000000001,"15min":419.22800000000001},"last":88.980999999999995}]},{"osd":6,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":16.931999999999999,"5min":21.992000000000001,"15min":19.459},"min":{"1min":6.6079999999999997,"5min":6.327,"15min":6.327},"max":{"1min":154.92400000000001,"5min":357.20499999999998,"15min":357.20499999999998},"last":15.124000000000001},{"interface":"front","average":{"1min":20.251999999999999,"5min":27.460999999999999,"15min":23.411000000000001},"min":{"1min":10.877000000000001,"5min":9.5079999999999991,"15min":9.5079999999999991},"max":{"1min":155.994,"5min":359.68099999999998,"15min":359.68099999999998},"last":14.048}]},{"osd":7,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.683,"5min":35.137999999999998,"15min":23.725999999999999},"min":{"1min":10.130000000000001,"5min":8.5199999999999996,"15min":7.5919999999999996},"max":{"1min":92.168999999999997,"5min":563.29700000000003,"15min":563.29700000000003},"last":12.036},{"interface":"front","average":{"1min":17.949000000000002,"5min":30.442,"15min":21.349},"min":{"1min":8.2240000000000002,"5min":8.2240000000000002,"15min":7.2919999999999998},"max":{"1min":113.15900000000001,"5min":352.65199999999999,"15min":352.65199999999999},"last":9.9580000000000002}]}]},{"osd":0,"up_from":1002,"seq":4303557231028,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60024,"kb_used_data":6472,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311816,"statfs":{"total":96636764160,"available":96575299584,"internally_reserved":0,"allocated":6627328,"data_stored":3587977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":77,"apply_latency_ms":77,"commit_latency_ns":77000000,"apply_latency_ns":77000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":71.099999999999994,"5min":1023.201,"15min":351.10899999999998},"min":{"1min":8.9689999999999994,"5min":5.9249999999999998,"15min":4.6269999999999998},"max":{"1min":350.00200000000001,"5min":27534,"15min":27534},"last":6.1740000000000004},{"interface":"front","average":{"1min":99.875,"5min":1025.7080000000001,"15min":350.98899999999998},"min":{"1min":5.9589999999999996,"5min":5.7240000000000002,"15min":5.0739999999999998},"max":{"1min":591.69600000000003,"5min":27535.008999999998,"15min":27535.008999999998},"last":6.6829999999999998}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.170999999999999,"5min":22.285,"15min":18.638000000000002},"min":{"1min":11.680999999999999,"5min":8.1069999999999993,"15min":6.6159999999999997},"max":{"1min":364.62799999999999,"5min":364.62799999999999,"15min":364.62799999999999},"last":9.9009999999999998},{"interface":"front","average":{"1min":45.890000000000001,"5min":26.417999999999999,"15min":21.443000000000001},"min":{"1min":14.172000000000001,"5min":10.023999999999999,"15min":8.8780000000000001},"max":{"1min":370.34300000000002,"5min":370.34300000000002,"15min":370.34300000000002},"last":11.736000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":66.832999999999998,"5min":25.178000000000001,"15min":355.89100000000002},"min":{"1min":6.7610000000000001,"5min":6.4409999999999998,"15min":5.891},"max":{"1min":444.59500000000003,"5min":444.59500000000003,"15min":30170.040000000001},"last":96.820999999999998},{"interface":"front","average":{"1min":65.754000000000005,"5min":33.606000000000002,"15min":363.07400000000001},"min":{"1min":10.266999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"max":{"1min":386.06099999999998,"5min":386.06099999999998,"15min":30171.194},"last":74.893000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":92.555000000000007,"5min":36.014000000000003,"15min":585.43899999999996},"min":{"1min":12.201000000000001,"5min":6.4480000000000004,"15min":6.0979999999999999},"max":{"1min":445.65300000000002,"5min":445.65300000000002,"15min":29892.669999999998},"last":11.266},{"interface":"front","average":{"1min":115.806,"5min":40.560000000000002,"15min":588.88900000000001},"min":{"1min":7.923,"5min":5.9400000000000004,"15min":5.9400000000000004},"max":{"1min":593.13499999999999,"5min":593.13499999999999,"15min":29877.289000000001},"last":14.489000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.341000000000001,"5min":28.972000000000001,"15min":56.932000000000002},"min":{"1min":12.702,"5min":9.5700000000000003,"15min":9.5700000000000003},"max":{"1min":403.13999999999999,"5min":403.13999999999999,"15min":508.02100000000002},"last":94.206999999999994},{"interface":"front","average":{"1min":43.966999999999999,"5min":23.216000000000001,"15min":32.374000000000002},"min":{"1min":7.6429999999999998,"5min":7.1210000000000004,"15min":7.1210000000000004},"max":{"1min":590.58500000000004,"5min":590.58500000000004,"15min":590.58500000000004},"last":92.817999999999998}]},{"osd":6,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":52.365000000000002,"5min":30.853999999999999,"15min":25.227},"min":{"1min":16.876000000000001,"5min":10.409000000000001,"15min":9.9350000000000005},"max":{"1min":375.36500000000001,"5min":375.36500000000001,"15min":375.36500000000001},"last":51.311},{"interface":"front","average":{"1min":55.634,"5min":32.579999999999998,"15min":27.417999999999999},"min":{"1min":18.984999999999999,"5min":11.564,"15min":11.564},"max":{"1min":380.31599999999997,"5min":380.31599999999997,"15min":380.31599999999997},"last":71.132999999999996}]},{"osd":7,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":57.960999999999999,"5min":28.638000000000002,"15min":20.780000000000001},"min":{"1min":17.748999999999999,"5min":8.4939999999999998,"15min":7.9349999999999996},"max":{"1min":385.01900000000001,"5min":385.01900000000001,"15min":385.01900000000001},"last":13.416},{"interface":"front","average":{"1min":54.853999999999999,"5min":29.765999999999998,"15min":25.545000000000002},"min":{"1min":10.992000000000001,"5min":8.1229999999999993,"15min":8.1229999999999993},"max":{"1min":446.69099999999997,"5min":446.69099999999997,"15min":446.69099999999997},"last":15.956}]}]},{"osd":4,"up_from":16,"seq":68719477161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60104,"kb_used_data":6912,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311736,"statfs":{"total":96636764160,"available":96575217664,"internally_reserved":0,"allocated":7077888,"data_stored":3907142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":13.432,"5min":21.457000000000001,"15min":87.253},"min":{"1min":6.4029999999999996,"5min":4.6230000000000002,"15min":4.6230000000000002},"max":{"1min":48.048999999999999,"5min":495.86900000000003,"15min":10627.831},"last":11.238},{"interface":"front","average":{"1min":18.010000000000002,"5min":25.013000000000002,"15min":90.852000000000004},"min":{"1min":7.2400000000000002,"5min":6.0990000000000002,"15min":5.6660000000000004},"max":{"1min":124.64100000000001,"5min":446.84100000000001,"15min":10653.877},"last":7.7329999999999997}]},{"osd":1,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":11.276999999999999,"5min":1441.567,"15min":495.98200000000003},"min":{"1min":7.2640000000000002,"5min":7.2640000000000002,"15min":5.9909999999999997},"max":{"1min":21.369,"5min":26872.857,"15min":26872.857},"last":8.4139999999999997},{"interface":"front","average":{"1min":11.013,"5min":1440.0119999999999,"15min":493.40499999999997},"min":{"1min":7.4269999999999996,"5min":7.3090000000000002,"15min":5.6909999999999998},"max":{"1min":17.440999999999999,"5min":26874.048999999999,"15min":26874.048999999999},"last":10.928000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":12.445,"5min":24.050999999999998,"15min":23.050000000000001},"min":{"1min":8.9000000000000004,"5min":7.3019999999999996,"15min":7.3019999999999996},"max":{"1min":15.904999999999999,"5min":453.40100000000001,"15min":597.11800000000005},"last":17.305},{"interface":"front","average":{"1min":15.029999999999999,"5min":27.077999999999999,"15min":26.128},"min":{"1min":12.429,"5min":11.631,"15min":8.0559999999999992},"max":{"1min":19.111000000000001,"5min":453.96199999999999,"15min":705.04499999999996},"last":57.628}]},{"osd":3,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":14.789999999999999,"5min":27.626999999999999,"15min":590.51199999999994},"min":{"1min":9.7219999999999995,"5min":7.6799999999999997,"15min":5.9740000000000002},"max":{"1min":23.942,"5min":448.65100000000001,"15min":30725.612000000001},"last":13.273999999999999},{"interface":"front","average":{"1min":12.497,"5min":20.047999999999998,"15min":582.06899999999996},"min":{"1min":7.7690000000000001,"5min":6.3840000000000003,"15min":6.3840000000000003},"max":{"1min":19.815000000000001,"5min":336.62599999999998,"15min":30407.659},"last":15.597}]},{"osd":5,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":16.190999999999999,"5min":30.579000000000001,"15min":32.869},"min":{"1min":14.1,"5min":7.1440000000000001,"15min":7.1440000000000001},"max":{"1min":18.738,"5min":450.81700000000001,"15min":450.81700000000001},"last":14.425000000000001},{"interface":"front","average":{"1min":10.430999999999999,"5min":19.959,"15min":22.803999999999998},"min":{"1min":6.2779999999999996,"5min":6.2779999999999996,"15min":6.2779999999999996},"max":{"1min":20.254999999999999,"5min":340.04000000000002,"15min":340.04000000000002},"last":7.9279999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":13.007,"5min":22.312000000000001,"15min":22.866},"min":{"1min":8.1120000000000001,"5min":7.1689999999999996,"15min":6.1520000000000001},"max":{"1min":22.010000000000002,"5min":339.47199999999998,"15min":696.952},"last":10.199999999999999},{"interface":"front","average":{"1min":14.724,"5min":23.917999999999999,"15min":23.209},"min":{"1min":9.0289999999999999,"5min":6.3449999999999998,"15min":6.3449999999999998},"max":{"1min":20.968,"5min":498.26999999999998,"15min":598.86900000000003},"last":16.390999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":12.805,"5min":24.097000000000001,"15min":24.802},"min":{"1min":8.7430000000000003,"5min":8.3399999999999999,"15min":7.7930000000000001},"max":{"1min":18.638000000000002,"5min":340.61900000000003,"15min":699.31799999999998},"last":13.837999999999999},{"interface":"front","average":{"1min":17.483000000000001,"5min":31.318999999999999,"15min":33.040999999999997},"min":{"1min":13.861000000000001,"5min":11.067,"15min":11.067},"max":{"1min":19.817,"5min":455.339,"15min":703.68200000000002},"last":11.691000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82276,"kb_used_data":6940,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94289564,"statfs":{"total":96636764160,"available":96552513536,"internally_reserved":0,"allocated":7106560,"data_stored":3935814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":12.891999999999999,"5min":16.984000000000002,"15min":62.423000000000002},"min":{"1min":6.9390000000000001,"5min":5.2350000000000003,"15min":5.2350000000000003},"max":{"1min":29.821000000000002,"5min":224.93899999999999,"15min":9502.5869999999995},"last":41.209000000000003},{"interface":"front","average":{"1min":15.925000000000001,"5min":19.867000000000001,"15min":65.921999999999997},"min":{"1min":8.2050000000000001,"5min":6.1580000000000004,"15min":6.1580000000000004},"max":{"1min":31.303999999999998,"5min":130.48400000000001,"15min":9511.3719999999994},"last":64.659000000000006}]},{"osd":1,"last update":"Sat Jul 29 14:04:49 2023","interfaces":[{"interface":"back","average":{"1min":17.140999999999998,"5min":970.20899999999995,"15min":335.46300000000002},"min":{"1min":7.5839999999999996,"5min":7.3369999999999997,"15min":4.8529999999999998},"max":{"1min":73.679000000000002,"5min":26903.921999999999,"15min":26903.921999999999},"last":8.5850000000000009},{"interface":"front","average":{"1min":12.878,"5min":963.24699999999996,"15min":332.12599999999998},"min":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.3330000000000002},"max":{"1min":69.432000000000002,"5min":26893.022000000001,"15min":26893.022000000001},"last":5.3620000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":14.891999999999999,"5min":21.295999999999999,"15min":19.027000000000001},"min":{"1min":10.173,"5min":7.8609999999999998,"15min":6.6299999999999999},"max":{"1min":26.302,"5min":191.73099999999999,"15min":191.73099999999999},"last":26.302},{"interface":"front","average":{"1min":11.848000000000001,"5min":16.003,"15min":15.422000000000001},"min":{"1min":7.4690000000000003,"5min":5.5499999999999998,"15min":5.5499999999999998},"max":{"1min":18,"5min":189.40199999999999,"15min":189.40199999999999},"last":11.739000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":10.653,"5min":15.525,"15min":521.48800000000006},"min":{"1min":7.2619999999999996,"5min":6.335,"15min":6.335},"max":{"1min":23.292000000000002,"5min":187.852,"15min":30710.619999999999},"last":37.712000000000003},{"interface":"front","average":{"1min":15.202999999999999,"5min":17.899000000000001,"15min":521.02300000000002},"min":{"1min":9.907,"5min":7.7009999999999996,"15min":7.6180000000000003},"max":{"1min":40.335000000000001,"5min":101.88200000000001,"15min":30591.776999999998},"last":38.539999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:56 2023","interfaces":[{"interface":"back","average":{"1min":16.030000000000001,"5min":808.61300000000006,"15min":1019.547},"min":{"1min":8.0950000000000006,"5min":6.718,"15min":6.1070000000000002},"max":{"1min":73.055999999999997,"5min":25620.438999999998,"15min":32899.582000000002},"last":8.2870000000000008},{"interface":"front","average":{"1min":16.128,"5min":828.35799999999995,"15min":1009.853},"min":{"1min":10.254,"5min":6.2089999999999996,"15min":6.2089999999999996},"max":{"1min":40.786999999999999,"5min":25979.712,"15min":32485.278999999999},"last":9.7769999999999992}]},{"osd":5,"last update":"Sat Jul 29 14:05:04 2023","interfaces":[{"interface":"back","average":{"1min":17.489999999999998,"5min":21.376999999999999,"15min":24.847000000000001},"min":{"1min":11.52,"5min":9.4949999999999992,"15min":8.9290000000000003},"max":{"1min":34.677,"5min":75.25,"15min":189.423},"last":94.355000000000004},{"interface":"front","average":{"1min":14.324,"5min":18.073,"15min":21.690000000000001},"min":{"1min":8.6969999999999992,"5min":7.3330000000000002,"15min":7.3330000000000002},"max":{"1min":24.77,"5min":80.085999999999999,"15min":188.518},"last":91.700999999999993}]},{"osd":7,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":24.321000000000002,"5min":19.489999999999998,"15min":19.814},"min":{"1min":14.164,"5min":7.1059999999999999,"15min":7.1059999999999999},"max":{"1min":74.337000000000003,"5min":74.337000000000003,"15min":200.55099999999999},"last":13.702999999999999},{"interface":"front","average":{"1min":18.581,"5min":20.780000000000001,"15min":22.265000000000001},"min":{"1min":9.6880000000000006,"5min":8.0790000000000006,"15min":8.0790000000000006},"max":{"1min":47.313000000000002,"5min":89.125,"15min":222.65199999999999},"last":12.701000000000001}]}]},{"osd":5,"up_from":704,"seq":3023656976772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78444,"kb_used_data":7204,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94293396,"statfs":{"total":96636764160,"available":96556437504,"internally_reserved":0,"allocated":7376896,"data_stored":4202582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":78,"apply_latency_ms":78,"commit_latency_ns":78000000,"apply_latency_ns":78000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:04:00 2023","interfaces":[{"interface":"back","average":{"1min":23.48,"5min":22.623000000000001,"15min":41.000999999999998},"min":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":4.742},"max":{"1min":150.63,"5min":262.56099999999998,"15min":299.87799999999999},"last":9.6039999999999992},{"interface":"front","average":{"1min":24.837,"5min":22.727,"15min":52.545999999999999},"min":{"1min":5.8259999999999996,"5min":4.6539999999999999,"15min":4.6539999999999999},"max":{"1min":111.14100000000001,"5min":253.80000000000001,"15min":370.00999999999999},"last":7.3899999999999997}]},{"osd":1,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":13.867000000000001,"5min":25.277000000000001,"15min":33.274999999999999},"min":{"1min":6.5419999999999998,"5min":5.3780000000000001,"15min":5.3780000000000001},"max":{"1min":91.590000000000003,"5min":362.58800000000002,"15min":362.58800000000002},"last":7.0529999999999999},{"interface":"front","average":{"1min":15.465999999999999,"5min":34.079000000000001,"15min":56.872999999999998},"min":{"1min":7.5650000000000004,"5min":6.8680000000000003,"15min":6.8680000000000003},"max":{"1min":85.760000000000005,"5min":356.69900000000001,"15min":363.50999999999999},"last":9.3510000000000009}]},{"osd":2,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.152000000000001,"5min":26.193000000000001,"15min":51.703000000000003},"min":{"1min":8.1880000000000006,"5min":7.032,"15min":7.032},"max":{"1min":151.804,"5min":312.755,"15min":384.024},"last":6.0259999999999998},{"interface":"front","average":{"1min":16.754000000000001,"5min":20.221,"15min":25.251000000000001},"min":{"1min":7.782,"5min":7.2759999999999998,"15min":7.2709999999999999},"max":{"1min":102.09099999999999,"5min":228.55600000000001,"15min":228.55600000000001},"last":11.063000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":28.802,"15min":55.957999999999998},"min":{"1min":10.734,"5min":7.4100000000000001,"15min":7.4100000000000001},"max":{"1min":131.36600000000001,"5min":312.12200000000001,"15min":381.23200000000003},"last":12.308},{"interface":"front","average":{"1min":18.568999999999999,"5min":28.875,"15min":66.894999999999996},"min":{"1min":7.944,"5min":7.944,"15min":7.944},"max":{"1min":118.456,"5min":265.92700000000002,"15min":382.779},"last":24.484999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":19.385999999999999,"5min":1108.5630000000001,"15min":412.19600000000003},"min":{"1min":10.5,"5min":7.343,"15min":7.3070000000000004},"max":{"1min":103.768,"5min":23572.741999999998,"15min":23572.741999999998},"last":8.423},{"interface":"front","average":{"1min":20.039000000000001,"5min":1113.885,"15min":428.73500000000001},"min":{"1min":8.7260000000000009,"5min":6.8929999999999998,"15min":6.8929999999999998},"max":{"1min":137.57400000000001,"5min":23610.142,"15min":23610.142},"last":10.224}]},{"osd":6,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":17.074000000000002,"5min":25.704999999999998,"15min":51.972000000000001},"min":{"1min":9.7360000000000007,"5min":8.1709999999999994,"15min":8.1709999999999994},"max":{"1min":41.347000000000001,"5min":263.983,"15min":450.17500000000001},"last":25.594000000000001},{"interface":"front","average":{"1min":18.436,"5min":26.684999999999999,"15min":57.575000000000003},"min":{"1min":11.185,"5min":8.1969999999999992,"15min":8.1969999999999992},"max":{"1min":63.707999999999998,"5min":221.99199999999999,"15min":417.303},"last":14.269}]},{"osd":7,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":21.550999999999998,"5min":28.064,"15min":47.530999999999999},"min":{"1min":12.340999999999999,"5min":11.222,"15min":10.602},"max":{"1min":104.718,"5min":284.48000000000002,"15min":341.30500000000001},"last":12.962999999999999},{"interface":"front","average":{"1min":18.460999999999999,"5min":27.806000000000001,"15min":53.993000000000002},"min":{"1min":12.089,"5min":11.042999999999999,"15min":11.042999999999999},"max":{"1min":46.353999999999999,"5min":191.33099999999999,"15min":329.86500000000001},"last":28.317}]}]},{"osd":7,"up_from":650,"seq":2791728742800,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60024,"kb_used_data":6472,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311816,"statfs":{"total":96636764160,"available":96575299584,"internally_reserved":0,"allocated":6627328,"data_stored":3587977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:45 2023","interfaces":[{"interface":"back","average":{"1min":11.94,"5min":13.635,"15min":23.015000000000001},"min":{"1min":5.5890000000000004,"5min":5.4539999999999997,"15min":5.4539999999999997},"max":{"1min":28.5,"5min":75.968000000000004,"15min":293.94499999999999},"last":11.478999999999999},{"interface":"front","average":{"1min":14.968999999999999,"5min":18.658999999999999,"15min":28.620000000000001},"min":{"1min":6.8179999999999996,"5min":6.8179999999999996,"15min":6.8179999999999996},"max":{"1min":29.143999999999998,"5min":116.952,"15min":358.267},"last":7.9210000000000003}]},{"osd":1,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":19.048999999999999,"5min":1108.855,"15min":389.67099999999999},"min":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":4.6660000000000004},"max":{"1min":184.25800000000001,"5min":27546.963,"15min":27546.963},"last":12.223000000000001},{"interface":"front","average":{"1min":22.969999999999999,"5min":1111.8620000000001,"15min":385.10300000000001},"min":{"1min":8.6110000000000007,"5min":5.5010000000000003,"15min":5.5010000000000003},"max":{"1min":150.75700000000001,"5min":27546.280999999999,"15min":27546.280999999999},"last":14.646000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.088000000000001,"5min":19.381,"15min":28.02},"min":{"1min":10.154999999999999,"5min":7.9100000000000001,"15min":7.9100000000000001},"max":{"1min":103.09,"5min":103.09,"15min":461.83800000000002},"last":12.978999999999999},{"interface":"front","average":{"1min":21.068000000000001,"5min":15.704000000000001,"15min":23.861999999999998},"min":{"1min":7.1269999999999998,"5min":6.6239999999999997,"15min":6.6239999999999997},"max":{"1min":181.97900000000001,"5min":181.97900000000001,"15min":385.40800000000002},"last":8.9149999999999991}]},{"osd":3,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":25.065999999999999,"5min":21.029,"15min":1531.4280000000001},"min":{"1min":7.976,"5min":7.6660000000000004,"15min":7.6660000000000004},"max":{"1min":242.197,"5min":242.197,"15min":34550.838000000003},"last":16.02},{"interface":"front","average":{"1min":31.428999999999998,"5min":27.873000000000001,"15min":1543.2929999999999},"min":{"1min":12.592000000000001,"5min":6.9550000000000001,"15min":6.9550000000000001},"max":{"1min":240.46899999999999,"5min":243.06399999999999,"15min":34556.381000000001},"last":18.369}]},{"osd":4,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":20.402999999999999,"5min":1074.817,"15min":900.84500000000003},"min":{"1min":6.7220000000000004,"5min":6.3719999999999999,"15min":6.3719999999999999},"max":{"1min":103.931,"5min":25146.41,"15min":27891.68},"last":9.7100000000000009},{"interface":"front","average":{"1min":29.045999999999999,"5min":1099.6300000000001,"15min":918.40200000000004},"min":{"1min":9.7360000000000007,"5min":8.1630000000000003,"15min":7.2699999999999996},"max":{"1min":189.47900000000001,"5min":25434.151999999998,"15min":27883.419999999998},"last":21.300999999999998}]},{"osd":5,"last update":"Sat Jul 29 14:05:08 2023","interfaces":[{"interface":"back","average":{"1min":17.529,"5min":28.314,"15min":72.040999999999997},"min":{"1min":14.367000000000001,"5min":11.215,"15min":10.683},"max":{"1min":22.012,"5min":230.16800000000001,"15min":1265.123},"last":21.904},{"interface":"front","average":{"1min":12.343999999999999,"5min":25.687999999999999,"15min":76.659000000000006},"min":{"1min":9.609,"5min":7.71,"15min":7.5540000000000003},"max":{"1min":21.030999999999999,"5min":255.23699999999999,"15min":1234.9300000000001},"last":15.369999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":21.149999999999999,"15min":25.884},"min":{"1min":10.222,"5min":9.9740000000000002,"15min":7.8689999999999998},"max":{"1min":78.843000000000004,"5min":95.558999999999997,"15min":243.66499999999999},"last":16.655000000000001},{"interface":"front","average":{"1min":24.378,"5min":25.236999999999998,"15min":32.558},"min":{"1min":12.111000000000001,"5min":10.542,"15min":10.542},"max":{"1min":92.266999999999996,"5min":148.334,"15min":454.68799999999999},"last":19.009}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-29T14:05:40.164 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-29T14:05:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.273+0000 7fedb2af5640 10 monclient: get_monmap_and_config 2023-07-29T14:05:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.273+0000 7fedb2af5640 10 monclient: build_initial_monmap 2023-07-29T14:05:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.273+0000 7fedb2af5640 10 monclient: monmap: 2023-07-29T14:05:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:40.278 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:40.278 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:40.275085+0000 2023-07-29T14:05:40.278 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:40.275085+0000 2023-07-29T14:05:40.278 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:40.278 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.278 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:40.281 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:40.281 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:40.281 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:40.281 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.274+0000 7fedb2af5640 1 Processor -- start 2023-07-29T14:05:40.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.274+0000 7fedb2af5640 1 -- start start 2023-07-29T14:05:40.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.274+0000 7fedb2af5640 10 monclient: init 2023-07-29T14:05:40.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:40.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2023-07-29T14:05:40.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fedac0b74a0 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:40.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fedac13e4f0 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:40.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fedac0aa150 0x7fedac144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:40.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 10 monclient(hunting): picked mon.noname-a con 0x7fedac0aa150 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:40.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedac0b7870 con 0x7fedac0b74a0 2023-07-29T14:05:40.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedac145230 con 0x7fedac13e4f0 2023-07-29T14:05:40.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:40.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedac0a91b0 con 0x7fedac0aa150 2023-07-29T14:05:40.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:40.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb2af5640 10 monclient(hunting): authenticate will time out at 3662.408365s 2023-07-29T14:05:40.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.275+0000 7fedb1092640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fedac0aa150 0x7fedac144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:40.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.276+0000 7fedabfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fedac13e4f0 0x7fedac1445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.40:35186/0 (socket says 172.21.15.40:35186) 2023-07-29T14:05:40.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.276+0000 7fedabfff640 1 -- 172.21.15.40:0/3255368482 learned_addr learned my addr 172.21.15.40:0/3255368482 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:40.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.276+0000 7fedb1092640 10 monclient(hunting): get_auth_request con 0x7fedac0aa150 auth_method 0 2023-07-29T14:05:40.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.276+0000 7fedb1092640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:40.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.276+0000 7fedb1092640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:40.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.276+0000 7fedb1092640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.277+0000 7fedb1092640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.277+0000 7fedb1092640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.277+0000 7fedb1092640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.278+0000 7fedab7fe640 1 -- 172.21.15.40:0/3255368482 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077286667 0 0) 0x7fedac145230 con 0x7fedac13e4f0 2023-07-29T14:05:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.278+0000 7fedab7fe640 10 monclient(hunting): _negotiate 2023-07-29T14:05:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.278+0000 7fedab7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.278+0000 7fedab7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:40.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.278+0000 7fedab7fe640 10 monclient(hunting): my global_id is 20314 2023-07-29T14:05:40.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.278+0000 7fedab7fe640 1 -- 172.21.15.40:0/3255368482 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed98003520 con 0x7fedac13e4f0 2023-07-29T14:05:40.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.279+0000 7fedb1092640 10 monclient(hunting): handle_auth_done global_id 14862 payload 274 2023-07-29T14:05:40.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.279+0000 7fedb1092640 1 -- 172.21.15.40:0/3255368482 >> v1:172.21.15.45:6789/0 conn(0x7fedac13e4f0 legacy=0x7fedac1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.279+0000 7fedb1092640 1 -- 172.21.15.40:0/3255368482 >> v1:172.21.15.40:6789/0 conn(0x7fedac0b74a0 legacy=0x7fedac13dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.279+0000 7fedb1092640 10 monclient: _finish_hunting 0 2023-07-29T14:05:40.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.279+0000 7fedb1092640 1 monclient: found mon.noname-a 2023-07-29T14:05:40.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.279+0000 7fedb1092640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:40.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.279+0000 7fedb1092640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T14:05:40.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.279+0000 7fedb1092640 1 -- 172.21.15.40:0/3255368482 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac0a9490 con 0x7fedac0aa150 2023-07-29T14:05:40.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.282+0000 7fedb1092640 1 --2- 172.21.15.40:0/3255368482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedac0aa150 0x7fedac144cf0 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0097a0 tx=0x7fed9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c580fc481be834 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:40.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 1 -- 172.21.15.40:0/3255368482 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c031830 con 0x7fedac0aa150 2023-07-29T14:05:40.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:40.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T14:05:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 10 monclient: dump: 2023-07-29T14:05:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.294 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:40.294 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:40.294 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:40.294 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 10 monclient: _finish_auth 0 2023-07-29T14:05:40.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 10 monclient: _check_auth_tickets 2023-07-29T14:05:40.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 1 -- 172.21.15.40:0/3255368482 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c031990 con 0x7fedac0aa150 2023-07-29T14:05:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedb2af5640 5 monclient: authenticate success, global_id 14862 2023-07-29T14:05:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedb2af5640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T14:05:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 1 -- 172.21.15.40:0/3255368482 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c031d20 con 0x7fedac0aa150 2023-07-29T14:05:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedb2af5640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedb2af5640 4 set_mon_vals no callback set 2023-07-29T14:05:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.287+0000 7fedab7fe640 10 monclient: dump: 2023-07-29T14:05:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:40.306 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:40.306 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.288+0000 7fedb2af5640 10 monclient: shutdown 2023-07-29T14:05:40.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.288+0000 7fedb2af5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:40.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.288+0000 7fedb2af5640 1 -- 172.21.15.40:0/3255368482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedac0aa150 msgr2=0x7fedac144cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.288+0000 7fedb2af5640 1 --2- 172.21.15.40:0/3255368482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedac0aa150 0x7fedac144cf0 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0097a0 tx=0x7fed9c02b8d0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.288+0000 7fedb2af5640 1 -- 172.21.15.40:0/3255368482 shutdown_connections 2023-07-29T14:05:40.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.288+0000 7fedb2af5640 1 --2- 172.21.15.40:0/3255368482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedac0aa150 0x7fedac144cf0 unknown :-1 s=CLOSED pgs=3200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.288+0000 7fedb2af5640 1 -- 172.21.15.40:0/3255368482 >> 172.21.15.40:0/3255368482 conn(0x7fedac065b00 msgr2=0x7fedac065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:40.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.288+0000 7fedb2af5640 1 -- 172.21.15.40:0/3255368482 shutdown_connections 2023-07-29T14:05:40.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.288+0000 7fedb2af5640 1 -- 172.21.15.40:0/3255368482 wait complete. 2023-07-29T14:05:40.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.289+0000 7fedb2af5640 10 monclient: build_initial_monmap 2023-07-29T14:05:40.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.289+0000 7fedb2af5640 10 monclient: monmap: 2023-07-29T14:05:40.308 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:40.308 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:40.308 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:40.308 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:40.309 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:40.310 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.310 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:40.310 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:40.310 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:40.310 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.289+0000 7fedb2af5640 1 librados: starting msgr at 2023-07-29T14:05:40.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.289+0000 7fedb2af5640 1 librados: starting objecter 2023-07-29T14:05:40.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.289+0000 7fedb2af5640 1 Processor -- start 2023-07-29T14:05:40.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.289+0000 7fedb2af5640 1 -- start start 2023-07-29T14:05:40.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.289+0000 7fedb2af5640 1 librados: setting wanted keys 2023-07-29T14:05:40.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.289+0000 7fedb2af5640 1 librados: calling monclient init 2023-07-29T14:05:40.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.289+0000 7fedb2af5640 10 monclient: init 2023-07-29T14:05:40.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:40.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T14:05:40.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedac0aa150 0x7fedac0b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 10 monclient(hunting): picked mon.noname-b con 0x7fedac0aa150 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fedac0b74a0 0x7fedac13a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 10 monclient(hunting): picked mon.noname-c con 0x7fedac0b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedac13e4f0 0x7fedac1d59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 10 monclient(hunting): picked mon.noname-a con 0x7fedac13e4f0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0b3f90 con 0x7fedac13e4f0 2023-07-29T14:05:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0aaa70 con 0x7fedac0aa150 2023-07-29T14:05:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac139720 con 0x7fedac0b74a0 2023-07-29T14:05:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb2af5640 10 monclient(hunting): authenticate will time out at 3662.423496s 2023-07-29T14:05:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.290+0000 7fedb1092640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedac13e4f0 0x7fedac1d59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb0891640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedac0aa150 0x7fedac0b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:40.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedabfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fedac0b74a0 0x7fedac13a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:40.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb0891640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedac0aa150 0x7fedac0b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:57444/0 (socket says 172.21.15.40:57444) 2023-07-29T14:05:40.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb0891640 1 -- 172.21.15.40:0/3546889279 learned_addr learned my addr 172.21.15.40:0/3546889279 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:40.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb0891640 10 monclient(hunting): get_auth_request con 0x7fedac0aa150 auth_method 0 2023-07-29T14:05:40.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb0891640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:40.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb0891640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:40.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb0891640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb1092640 10 monclient(hunting): get_auth_request con 0x7fedac13e4f0 auth_method 0 2023-07-29T14:05:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb1092640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb1092640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb1092640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedabfff640 10 monclient(hunting): get_auth_request con 0x7fedac0b74a0 auth_method 0 2023-07-29T14:05:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedabfff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedabfff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb0891640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb0891640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.291+0000 7fedb0891640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.292+0000 7fedabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.292+0000 7fedabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.292+0000 7fedabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.292+0000 7fedb1092640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.292+0000 7fedb1092640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.292+0000 7fedb1092640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:40.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.294+0000 7fedabfff640 10 monclient(hunting): handle_auth_done global_id 16343 payload 824 2023-07-29T14:05:40.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.294+0000 7fedabfff640 1 -- 172.21.15.40:0/3546889279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedac0aa150 msgr2=0x7fedac0b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.294+0000 7fedabfff640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedac0aa150 0x7fedac0b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.294+0000 7fedabfff640 1 -- 172.21.15.40:0/3546889279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedac13e4f0 msgr2=0x7fedac1d59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.294+0000 7fedabfff640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedac13e4f0 0x7fedac1d59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.294+0000 7fedabfff640 10 monclient: _finish_hunting 0 2023-07-29T14:05:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.294+0000 7fedabfff640 1 monclient: found mon.noname-c 2023-07-29T14:05:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.294+0000 7fedabfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.294+0000 7fedabfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.294+0000 7fedabfff640 1 -- 172.21.15.40:0/3546889279 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c009400 con 0x7fedac0b74a0 2023-07-29T14:05:40.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.294+0000 7fedb0891640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedac0aa150 0x7fedac0b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-29T14:05:40.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.297+0000 7fedabfff640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fedac0b74a0 0x7fedac13a130 secure :-1 s=READY pgs=3217 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c420 tx=0x7fed94001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:40.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.297+0000 7feda97fa640 10 client.?.objecter ms_handle_connect 0x7fedac0b74a0 2023-07-29T14:05:40.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.297+0000 7feda97fa640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:40.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 1 -- 172.21.15.40:0/3546889279 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94005ce0 con 0x7fedac0b74a0 2023-07-29T14:05:40.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:40.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:40.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 10 monclient: dump: 2023-07-29T14:05:40.328 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:40.328 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:40.329 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.329 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.329 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:40.329 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.330 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:40.330 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:40.330 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:40.330 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 10 monclient: _finish_auth 0 2023-07-29T14:05:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 10 monclient: _check_auth_tickets 2023-07-29T14:05:40.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:40.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 1 -- 172.21.15.40:0/3546889279 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed94005e40 con 0x7fedac0b74a0 2023-07-29T14:05:40.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:40.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 1 -- 172.21.15.40:0/3546889279 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed940072d0 con 0x7fedac0b74a0 2023-07-29T14:05:40.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:40.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7feda97fa640 10 monclient: dump: 2023-07-29T14:05:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.339 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:40.340 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:40.340 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:40.341 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7fedb2af5640 5 monclient: authenticate success, global_id 16343 2023-07-29T14:05:40.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7fedb2af5640 10 monclient: _renew_subs 2023-07-29T14:05:40.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7fedb2af5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:40.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7fedb2af5640 1 -- 172.21.15.40:0/3546889279 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac1399a0 con 0x7fedac0b74a0 2023-07-29T14:05:40.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7fedaaffd640 4 set_mon_vals no callback set 2023-07-29T14:05:40.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7fedb2af5640 10 client.16343.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:40.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7fedb2af5640 10 monclient: _renew_subs 2023-07-29T14:05:40.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7fedb2af5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:40.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7fedb2af5640 1 -- 172.21.15.40:0/3546889279 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac1d60c0 con 0x7fedac0b74a0 2023-07-29T14:05:40.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.301+0000 7fedb2af5640 1 librados: init done 2023-07-29T14:05:40.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.302+0000 7fedb2af5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:40.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.302+0000 7fedb2af5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:40.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.302+0000 7fedb2af5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:40.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.302+0000 7fedb2af5640 1 -- 172.21.15.40:0/3546889279 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac0aac00 con 0x7fedac0b74a0 2023-07-29T14:05:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.561+0000 7feda97fa640 1 -- 172.21.15.40:0/3546889279 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fed94007430 con 0x7fedac0b74a0 2023-07-29T14:05:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.561+0000 7feda97fa640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fed8c06bfb0 0x7fed8c06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:40.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.561+0000 7fedb0891640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fed8c06bfb0 0x7fed8c06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:40.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.562+0000 7fedb0891640 10 monclient: get_auth_request con 0x7fed8c06bfb0 auth_method 0 2023-07-29T14:05:40.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.562+0000 7fedb0891640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fed8c06bfb0 0x7fed8c06e460 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feda400f290 tx=0x7feda4009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:40.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.562+0000 7feda97fa640 10 client.16343.objecter ms_handle_connect 0x7fed8c06bfb0 2023-07-29T14:05:40.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.575+0000 7feda97fa640 1 -- 172.21.15.40:0/3546889279 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7fed9405b580 con 0x7fedac0b74a0 2023-07-29T14:05:40.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.575+0000 7feda97fa640 10 client.16343.objecter ms_dispatch 0x7fedac000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:40.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.575+0000 7feda97fa640 3 client.16343.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:40.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.575+0000 7feda97fa640 3 client.16343.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:40.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.576+0000 7feda97fa640 20 client.16343.objecter dump_active .. 0 homeless 2023-07-29T14:05:40.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.576+0000 7feda97fa640 1 -- 172.21.15.40:0/3546889279 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7fed940539b0 con 0x7fedac0b74a0 2023-07-29T14:05:40.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.576+0000 7feda97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:40.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.576+0000 7feda97fa640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:40.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.750+0000 7fedb2af5640 1 -- 172.21.15.40:0/3546889279 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fedac0aa750 con 0x7fed8c06bfb0 2023-07-29T14:05:40.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.756+0000 7feda97fa640 1 -- 172.21.15.40:0/3546889279 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152067 (secure 0 0 0) 0x7feda4001fc0 con 0x7fed8c06bfb0 2023-07-29T14:05:40.760 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T14:05:40.762 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-29T14:05:40.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 10 librados: watch_flush enter 2023-07-29T14:05:40.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 10 librados: watch_flush exit 2023-07-29T14:05:40.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 20 client.16343.objecter shutdown clearing up homeless session... 2023-07-29T14:05:40.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 10 client.16343.objecter successfully canceled tick 2023-07-29T14:05:40.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 -- 172.21.15.40:0/3546889279 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fed8c06bfb0 msgr2=0x7fed8c06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fed8c06bfb0 0x7fed8c06e460 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feda400f290 tx=0x7feda4009040 comp rx=0 tx=0).stop 2023-07-29T14:05:40.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 10 monclient: shutdown 2023-07-29T14:05:40.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:40.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 -- 172.21.15.40:0/3546889279 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fedac0b74a0 msgr2=0x7fedac13a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fedac0b74a0 0x7fedac13a130 secure :-1 s=READY pgs=3217 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c420 tx=0x7fed94001e70 comp rx=0 tx=0).stop 2023-07-29T14:05:40.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 -- 172.21.15.40:0/3546889279 shutdown_connections 2023-07-29T14:05:40.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fed8c06bfb0 0x7fed8c06e460 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedac13e4f0 0x7fedac1d59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fedac0b74a0 0x7fedac13a130 unknown :-1 s=CLOSED pgs=3217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 --2- 172.21.15.40:0/3546889279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedac0aa150 0x7fedac0b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 -- 172.21.15.40:0/3546889279 >> 172.21.15.40:0/3546889279 conn(0x7fedac065b00 msgr2=0x7fedac0b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:40.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 -- 172.21.15.40:0/3546889279 shutdown_connections 2023-07-29T14:05:40.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 -- 172.21.15.40:0/3546889279 wait complete. 2023-07-29T14:05:40.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.758+0000 7fedb2af5640 1 librados: shutdown 2023-07-29T14:05:40.781 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":2131,"stamp":"2023-07-29T14:05:39.629046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":81,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":525432,"kb_used_data":54888,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754449288,"statfs":{"total":773094113280,"available":772556070912,"internally_reserved":0,"allocated":56205312,"data_stored":31064796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":452,"apply_latency_ms":452,"commit_latency_ns":452000000,"apply_latency_ns":452000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4000299,"num_objects":-44,"num_object_clones":-1,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44,"num_whiteouts":0,"num_read":-31,"num_read_kb":-11,"num_write":-71,"num_write_kb":-3969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.483953"},"pg_stats":[{"pgid":"7.1b","version":"0'0","reported_seq":968,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.762797+0000","last_change":"2023-07-29T13:56:55.662556+0000","last_active":"2023-07-29T14:05:36.762797+0000","last_peered":"2023-07-29T14:05:36.762797+0000","last_clean":"2023-07-29T14:05:36.762797+0000","last_became_active":"2023-07-29T13:56:55.656930+0000","last_became_peered":"2023-07-29T13:56:55.656930+0000","last_unstale":"2023-07-29T14:05:36.762797+0000","last_undegraded":"2023-07-29T14:05:36.762797+0000","last_fullsized":"2023-07-29T14:05:36.762797+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:52:51.321547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":977,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.016088+0000","last_change":"2023-07-29T13:41:38.328763+0000","last_active":"2023-07-29T14:05:37.016088+0000","last_peered":"2023-07-29T14:05:37.016088+0000","last_clean":"2023-07-29T14:05:37.016088+0000","last_became_active":"2023-07-29T13:41:38.322679+0000","last_became_peered":"2023-07-29T13:41:38.322679+0000","last_unstale":"2023-07-29T14:05:37.016088+0000","last_undegraded":"2023-07-29T14:05:37.016088+0000","last_fullsized":"2023-07-29T14:05:37.016088+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:40:55.886733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":964,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.997533+0000","last_change":"2023-07-29T13:29:52.105383+0000","last_active":"2023-07-29T14:05:36.997533+0000","last_peered":"2023-07-29T14:05:36.997533+0000","last_clean":"2023-07-29T14:05:36.997533+0000","last_became_active":"2023-07-29T13:29:52.099163+0000","last_became_peered":"2023-07-29T13:29:52.099163+0000","last_unstale":"2023-07-29T14:05:36.997533+0000","last_undegraded":"2023-07-29T14:05:36.997533+0000","last_fullsized":"2023-07-29T14:05:36.997533+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:03:36.535099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":930,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.709810+0000","last_change":"2023-07-29T13:29:41.816664+0000","last_active":"2023-07-29T14:05:36.709810+0000","last_peered":"2023-07-29T14:05:36.709810+0000","last_clean":"2023-07-29T14:05:36.709810+0000","last_became_active":"2023-07-29T13:29:41.782506+0000","last_became_peered":"2023-07-29T13:29:41.782506+0000","last_unstale":"2023-07-29T14:05:36.709810+0000","last_undegraded":"2023-07-29T14:05:36.709810+0000","last_fullsized":"2023-07-29T14:05:36.709810+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:20:57.566653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":947,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.066403+0000","last_change":"2023-07-29T14:05:38.066403+0000","last_active":"2023-07-29T14:05:38.066403+0000","last_peered":"2023-07-29T14:05:38.066403+0000","last_clean":"2023-07-29T14:05:38.066403+0000","last_became_active":"2023-07-29T14:05:38.060277+0000","last_became_peered":"2023-07-29T14:05:38.060277+0000","last_unstale":"2023-07-29T14:05:38.066403+0000","last_undegraded":"2023-07-29T14:05:38.066403+0000","last_fullsized":"2023-07-29T14:05:38.066403+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:50:25.119138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":913,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.286005+0000","last_change":"2023-07-29T13:53:19.606247+0000","last_active":"2023-07-29T14:05:36.286005+0000","last_peered":"2023-07-29T14:05:36.286005+0000","last_clean":"2023-07-29T14:05:36.286005+0000","last_became_active":"2023-07-29T13:53:19.600386+0000","last_became_peered":"2023-07-29T13:53:19.600386+0000","last_unstale":"2023-07-29T14:05:36.286005+0000","last_undegraded":"2023-07-29T14:05:36.286005+0000","last_fullsized":"2023-07-29T14:05:36.286005+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:46:03.055229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":934,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.436676+0000","last_change":"2023-07-29T13:56:55.406251+0000","last_active":"2023-07-29T14:05:34.436676+0000","last_peered":"2023-07-29T14:05:34.436676+0000","last_clean":"2023-07-29T14:05:34.436676+0000","last_became_active":"2023-07-29T13:56:55.400591+0000","last_became_peered":"2023-07-29T13:56:55.400591+0000","last_unstale":"2023-07-29T14:05:34.436676+0000","last_undegraded":"2023-07-29T14:05:34.436676+0000","last_fullsized":"2023-07-29T14:05:34.436676+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:40:48.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":954,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.369590+0000","last_change":"2023-07-29T13:29:31.089676+0000","last_active":"2023-07-29T14:05:36.369590+0000","last_peered":"2023-07-29T14:05:36.369590+0000","last_clean":"2023-07-29T14:05:36.369590+0000","last_became_active":"2023-07-29T13:29:31.072042+0000","last_became_peered":"2023-07-29T13:29:31.072042+0000","last_unstale":"2023-07-29T14:05:36.369590+0000","last_undegraded":"2023-07-29T14:05:36.369590+0000","last_fullsized":"2023-07-29T14:05:36.369590+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:28:28.141003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":912,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.243324+0000","last_change":"2023-07-29T13:53:19.677067+0000","last_active":"2023-07-29T14:05:36.243324+0000","last_peered":"2023-07-29T14:05:36.243324+0000","last_clean":"2023-07-29T14:05:36.243324+0000","last_became_active":"2023-07-29T13:53:19.671215+0000","last_became_peered":"2023-07-29T13:53:19.671215+0000","last_unstale":"2023-07-29T14:05:36.243324+0000","last_undegraded":"2023-07-29T14:05:36.243324+0000","last_fullsized":"2023-07-29T14:05:36.243324+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:51:18.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":985,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.445062+0000","last_change":"2023-07-29T13:56:55.517147+0000","last_active":"2023-07-29T14:05:35.445062+0000","last_peered":"2023-07-29T14:05:35.445062+0000","last_clean":"2023-07-29T14:05:35.445062+0000","last_became_active":"2023-07-29T13:56:55.511490+0000","last_became_peered":"2023-07-29T13:56:55.511490+0000","last_unstale":"2023-07-29T14:05:35.445062+0000","last_undegraded":"2023-07-29T14:05:35.445062+0000","last_fullsized":"2023-07-29T14:05:35.445062+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:26.433868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":962,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.507347+0000","last_change":"2023-07-29T14:05:37.515837+0000","last_active":"2023-07-29T14:05:36.327038+0000","last_peered":"2023-07-29T14:05:35.596454+0000","last_clean":"2023-07-29T14:04:22.472067+0000","last_became_active":"2023-07-29T14:05:29.733569+0000","last_became_peered":"2023-07-29T14:05:29.733569+0000","last_unstale":"2023-07-29T14:05:37.507347+0000","last_undegraded":"2023-07-29T14:05:37.507347+0000","last_fullsized":"2023-07-29T14:05:37.507347+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:03:53.476296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":913,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.470943+0000","last_change":"2023-07-29T13:53:19.644608+0000","last_active":"2023-07-29T14:05:37.470943+0000","last_peered":"2023-07-29T14:05:37.470943+0000","last_clean":"2023-07-29T14:05:37.470943+0000","last_became_active":"2023-07-29T13:53:19.637473+0000","last_became_peered":"2023-07-29T13:53:19.637473+0000","last_unstale":"2023-07-29T14:05:37.470943+0000","last_undegraded":"2023-07-29T14:05:37.470943+0000","last_fullsized":"2023-07-29T14:05:37.470943+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:53:02.384899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":932,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.746967+0000","last_change":"2023-07-29T13:29:33.530600+0000","last_active":"2023-07-29T14:05:36.746967+0000","last_peered":"2023-07-29T14:05:36.746967+0000","last_clean":"2023-07-29T14:05:36.746967+0000","last_became_active":"2023-07-29T13:29:33.515255+0000","last_became_peered":"2023-07-29T13:29:33.515255+0000","last_unstale":"2023-07-29T14:05:36.746967+0000","last_undegraded":"2023-07-29T14:05:36.746967+0000","last_fullsized":"2023-07-29T14:05:36.746967+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:21:56.794070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":979,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.433490+0000","last_change":"2023-07-29T14:05:37.441418+0000","last_active":"2023-07-29T14:05:36.969928+0000","last_peered":"2023-07-29T14:05:35.332022+0000","last_clean":"2023-07-29T14:05:12.448301+0000","last_became_active":"2023-07-29T14:05:29.733673+0000","last_became_peered":"2023-07-29T14:05:29.733673+0000","last_unstale":"2023-07-29T14:05:37.433490+0000","last_undegraded":"2023-07-29T14:05:37.433490+0000","last_fullsized":"2023-07-29T14:05:37.433490+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:38:04.875711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":936,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.418624+0000","last_change":"2023-07-29T13:56:55.146603+0000","last_active":"2023-07-29T14:05:34.418624+0000","last_peered":"2023-07-29T14:05:34.418624+0000","last_clean":"2023-07-29T14:05:34.418624+0000","last_became_active":"2023-07-29T13:56:55.139167+0000","last_became_peered":"2023-07-29T13:56:55.139167+0000","last_unstale":"2023-07-29T14:05:34.418624+0000","last_undegraded":"2023-07-29T14:05:34.418624+0000","last_fullsized":"2023-07-29T14:05:34.418624+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:56:23.477235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":944,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.533539+0000","last_change":"2023-07-29T14:05:37.544652+0000","last_active":"2023-07-29T14:05:36.720848+0000","last_peered":"2023-07-29T14:05:35.604442+0000","last_clean":"2023-07-29T14:04:06.556968+0000","last_became_active":"2023-07-29T14:05:29.660537+0000","last_became_peered":"2023-07-29T14:05:29.660537+0000","last_unstale":"2023-07-29T14:05:37.533539+0000","last_undegraded":"2023-07-29T14:05:37.533539+0000","last_fullsized":"2023-07-29T14:05:37.533539+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:42:16.790121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":987,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.436816+0000","last_change":"2023-07-29T13:56:55.073408+0000","last_active":"2023-07-29T14:05:35.436816+0000","last_peered":"2023-07-29T14:05:35.436816+0000","last_clean":"2023-07-29T14:05:35.436816+0000","last_became_active":"2023-07-29T13:56:55.065724+0000","last_became_peered":"2023-07-29T13:56:55.065724+0000","last_unstale":"2023-07-29T14:05:35.436816+0000","last_undegraded":"2023-07-29T14:05:35.436816+0000","last_fullsized":"2023-07-29T14:05:35.436816+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:42:58.825948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":971,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.050103+0000","last_change":"2023-07-29T13:28:14.434312+0000","last_active":"2023-07-29T14:05:37.050103+0000","last_peered":"2023-07-29T14:05:37.050103+0000","last_clean":"2023-07-29T14:05:37.050103+0000","last_became_active":"2023-07-29T13:28:14.423044+0000","last_became_peered":"2023-07-29T13:28:14.423044+0000","last_unstale":"2023-07-29T14:05:37.050103+0000","last_undegraded":"2023-07-29T14:05:37.050103+0000","last_fullsized":"2023-07-29T14:05:37.050103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:56:27.865715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1","version":"70'7","reported_seq":983,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.310515+0000","last_change":"2023-07-29T13:41:38.528285+0000","last_active":"2023-07-29T14:05:36.310515+0000","last_peered":"2023-07-29T14:05:36.310515+0000","last_clean":"2023-07-29T14:05:36.310515+0000","last_became_active":"2023-07-29T13:41:38.521847+0000","last_became_peered":"2023-07-29T13:41:38.521847+0000","last_unstale":"2023-07-29T14:05:36.310515+0000","last_undegraded":"2023-07-29T14:05:36.310515+0000","last_fullsized":"2023-07-29T14:05:36.310515+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:08:42.836378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":942,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.430995+0000","last_change":"2023-07-29T13:56:55.263959+0000","last_active":"2023-07-29T14:05:34.430995+0000","last_peered":"2023-07-29T14:05:34.430995+0000","last_clean":"2023-07-29T14:05:34.430995+0000","last_became_active":"2023-07-29T13:56:55.258241+0000","last_became_peered":"2023-07-29T13:56:55.258241+0000","last_unstale":"2023-07-29T14:05:34.430995+0000","last_undegraded":"2023-07-29T14:05:34.430995+0000","last_fullsized":"2023-07-29T14:05:34.430995+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:42.517469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":966,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.065287+0000","last_change":"2023-07-29T13:29:52.230014+0000","last_active":"2023-07-29T14:05:37.065287+0000","last_peered":"2023-07-29T14:05:37.065287+0000","last_clean":"2023-07-29T14:05:37.065287+0000","last_became_active":"2023-07-29T13:29:52.224467+0000","last_became_peered":"2023-07-29T13:29:52.224467+0000","last_unstale":"2023-07-29T14:05:37.065287+0000","last_undegraded":"2023-07-29T14:05:37.065287+0000","last_fullsized":"2023-07-29T14:05:37.065287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:51:03.019114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":972,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.014430+0000","last_change":"2023-07-29T13:28:14.652586+0000","last_active":"2023-07-29T14:05:37.014430+0000","last_peered":"2023-07-29T14:05:37.014430+0000","last_clean":"2023-07-29T14:05:37.014430+0000","last_became_active":"2023-07-29T13:28:14.642023+0000","last_became_peered":"2023-07-29T13:28:14.642023+0000","last_unstale":"2023-07-29T14:05:37.014430+0000","last_undegraded":"2023-07-29T14:05:37.014430+0000","last_fullsized":"2023-07-29T14:05:37.014430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:25:23.334956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":933,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.764552+0000","last_change":"2023-07-29T13:29:41.869711+0000","last_active":"2023-07-29T14:05:36.764552+0000","last_peered":"2023-07-29T14:05:36.764552+0000","last_clean":"2023-07-29T14:05:36.764552+0000","last_became_active":"2023-07-29T13:29:41.844314+0000","last_became_peered":"2023-07-29T13:29:41.844314+0000","last_unstale":"2023-07-29T14:05:36.764552+0000","last_undegraded":"2023-07-29T14:05:36.764552+0000","last_fullsized":"2023-07-29T14:05:36.764552+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:41:40.503015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":963,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.209438+0000","last_change":"2023-07-29T13:56:55.159956+0000","last_active":"2023-07-29T14:05:36.209438+0000","last_peered":"2023-07-29T14:05:36.209438+0000","last_clean":"2023-07-29T14:05:36.209438+0000","last_became_active":"2023-07-29T13:56:55.150514+0000","last_became_peered":"2023-07-29T13:56:55.150514+0000","last_unstale":"2023-07-29T14:05:36.209438+0000","last_undegraded":"2023-07-29T14:05:36.209438+0000","last_fullsized":"2023-07-29T14:05:36.209438+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:16:38.329000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1005,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.048452+0000","last_change":"2023-07-29T13:56:55.615681+0000","last_active":"2023-07-29T14:05:37.048452+0000","last_peered":"2023-07-29T14:05:37.048452+0000","last_clean":"2023-07-29T14:05:37.048452+0000","last_became_active":"2023-07-29T13:56:55.609414+0000","last_became_peered":"2023-07-29T13:56:55.609414+0000","last_unstale":"2023-07-29T14:05:37.048452+0000","last_undegraded":"2023-07-29T14:05:37.048452+0000","last_fullsized":"2023-07-29T14:05:37.048452+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:36:30.601005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1012,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.999968+0000","last_change":"2023-07-29T13:56:55.550112+0000","last_active":"2023-07-29T14:05:36.999968+0000","last_peered":"2023-07-29T14:05:36.999968+0000","last_clean":"2023-07-29T14:05:36.999968+0000","last_became_active":"2023-07-29T13:56:55.543861+0000","last_became_peered":"2023-07-29T13:56:55.543861+0000","last_unstale":"2023-07-29T14:05:36.999968+0000","last_undegraded":"2023-07-29T14:05:36.999968+0000","last_fullsized":"2023-07-29T14:05:36.999968+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:57:36.179121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":940,"reported_epoch":1000,"state":"active+undersized","last_fresh":"2023-07-29T14:05:34.441767+0000","last_change":"2023-07-29T14:05:29.759408+0000","last_active":"2023-07-29T14:05:34.441767+0000","last_peered":"2023-07-29T14:05:34.441767+0000","last_clean":"2023-07-29T14:04:06.482879+0000","last_became_active":"2023-07-29T14:05:29.759408+0000","last_became_peered":"2023-07-29T14:05:29.759408+0000","last_unstale":"2023-07-29T14:05:34.441767+0000","last_undegraded":"2023-07-29T14:05:34.441767+0000","last_fullsized":"2023-07-29T14:05:29.709468+0000","mapping_epoch":995,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:11:56.037546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":972,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.010320+0000","last_change":"2023-07-29T13:28:14.713485+0000","last_active":"2023-07-29T14:05:37.010320+0000","last_peered":"2023-07-29T14:05:37.010320+0000","last_clean":"2023-07-29T14:05:37.010320+0000","last_became_active":"2023-07-29T13:28:14.708042+0000","last_became_peered":"2023-07-29T13:28:14.708042+0000","last_unstale":"2023-07-29T14:05:37.010320+0000","last_undegraded":"2023-07-29T14:05:37.010320+0000","last_fullsized":"2023-07-29T14:05:37.010320+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:25:56.525512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":1099,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.438875+0000","last_change":"2023-07-29T13:56:55.164457+0000","last_active":"2023-07-29T14:05:35.438875+0000","last_peered":"2023-07-29T14:05:35.438875+0000","last_clean":"2023-07-29T14:05:35.438875+0000","last_became_active":"2023-07-29T13:56:55.157702+0000","last_became_peered":"2023-07-29T13:56:55.157702+0000","last_unstale":"2023-07-29T14:05:35.438875+0000","last_undegraded":"2023-07-29T14:05:35.438875+0000","last_fullsized":"2023-07-29T14:05:35.438875+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_clean_scrub_stamp":"2023-07-29T13:28:06.434639+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:06:43.352094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":995,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.367084+0000","last_change":"2023-07-29T13:56:55.347022+0000","last_active":"2023-07-29T14:05:36.367084+0000","last_peered":"2023-07-29T14:05:36.367084+0000","last_clean":"2023-07-29T14:05:36.367084+0000","last_became_active":"2023-07-29T13:56:55.341426+0000","last_became_peered":"2023-07-29T13:56:55.341426+0000","last_unstale":"2023-07-29T14:05:36.367084+0000","last_undegraded":"2023-07-29T14:05:36.367084+0000","last_fullsized":"2023-07-29T14:05:36.367084+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:46:20.874037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":945,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.466789+0000","last_change":"2023-07-29T13:41:38.063359+0000","last_active":"2023-07-29T14:05:35.466789+0000","last_peered":"2023-07-29T14:05:35.466789+0000","last_clean":"2023-07-29T14:05:35.466789+0000","last_became_active":"2023-07-29T13:41:38.057649+0000","last_became_peered":"2023-07-29T13:41:38.057649+0000","last_unstale":"2023-07-29T14:05:35.466789+0000","last_undegraded":"2023-07-29T14:05:35.466789+0000","last_fullsized":"2023-07-29T14:05:35.466789+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:36:46.868780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":925,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.290603+0000","last_change":"2023-07-29T13:53:19.566408+0000","last_active":"2023-07-29T14:05:36.290603+0000","last_peered":"2023-07-29T14:05:36.290603+0000","last_clean":"2023-07-29T14:05:36.290603+0000","last_became_active":"2023-07-29T13:53:19.560480+0000","last_became_peered":"2023-07-29T13:53:19.560480+0000","last_unstale":"2023-07-29T14:05:36.290603+0000","last_undegraded":"2023-07-29T14:05:36.290603+0000","last_fullsized":"2023-07-29T14:05:36.290603+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:01:49.170231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":913,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.314881+0000","last_change":"2023-07-29T13:53:19.617225+0000","last_active":"2023-07-29T14:05:36.314881+0000","last_peered":"2023-07-29T14:05:36.314881+0000","last_clean":"2023-07-29T14:05:36.314881+0000","last_became_active":"2023-07-29T13:53:19.611477+0000","last_became_peered":"2023-07-29T13:53:19.611477+0000","last_unstale":"2023-07-29T14:05:36.314881+0000","last_undegraded":"2023-07-29T14:05:36.314881+0000","last_fullsized":"2023-07-29T14:05:36.314881+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:52:56.727655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":948,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.411190+0000","last_change":"2023-07-29T13:56:55.182600+0000","last_active":"2023-07-29T14:05:34.411190+0000","last_peered":"2023-07-29T14:05:34.411190+0000","last_clean":"2023-07-29T14:05:34.411190+0000","last_became_active":"2023-07-29T13:56:55.173228+0000","last_became_peered":"2023-07-29T13:56:55.173228+0000","last_unstale":"2023-07-29T14:05:34.411190+0000","last_undegraded":"2023-07-29T14:05:34.411190+0000","last_fullsized":"2023-07-29T14:05:34.411190+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:57:08.005958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":935,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.452075+0000","last_change":"2023-07-29T13:56:55.243737+0000","last_active":"2023-07-29T14:05:34.452075+0000","last_peered":"2023-07-29T14:05:34.452075+0000","last_clean":"2023-07-29T14:05:34.452075+0000","last_became_active":"2023-07-29T13:56:55.238036+0000","last_became_peered":"2023-07-29T13:56:55.238036+0000","last_unstale":"2023-07-29T14:05:34.452075+0000","last_undegraded":"2023-07-29T14:05:34.452075+0000","last_fullsized":"2023-07-29T14:05:34.452075+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:04:27.494041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":931,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.745324+0000","last_change":"2023-07-29T13:29:33.527250+0000","last_active":"2023-07-29T14:05:36.745324+0000","last_peered":"2023-07-29T14:05:36.745324+0000","last_clean":"2023-07-29T14:05:36.745324+0000","last_became_active":"2023-07-29T13:29:33.508113+0000","last_became_peered":"2023-07-29T13:29:33.508113+0000","last_unstale":"2023-07-29T14:05:36.745324+0000","last_undegraded":"2023-07-29T14:05:36.745324+0000","last_fullsized":"2023-07-29T14:05:36.745324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:52:25.906655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":937,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.193360+0000","last_change":"2023-07-29T13:29:32.261353+0000","last_active":"2023-07-29T14:05:34.193360+0000","last_peered":"2023-07-29T14:05:34.193360+0000","last_clean":"2023-07-29T14:05:34.193360+0000","last_became_active":"2023-07-29T13:29:32.123788+0000","last_became_peered":"2023-07-29T13:29:32.123788+0000","last_unstale":"2023-07-29T14:05:34.193360+0000","last_undegraded":"2023-07-29T14:05:34.193360+0000","last_fullsized":"2023-07-29T14:05:34.193360+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:25:55.926424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":934,"reported_epoch":1000,"state":"active+clean","last_fresh":"2023-07-29T14:05:34.429328+0000","last_change":"2023-07-29T13:56:55.597264+0000","last_active":"2023-07-29T14:05:34.429328+0000","last_peered":"2023-07-29T14:05:34.429328+0000","last_clean":"2023-07-29T14:05:34.429328+0000","last_became_active":"2023-07-29T13:56:55.591631+0000","last_became_peered":"2023-07-29T13:56:55.591631+0000","last_unstale":"2023-07-29T14:05:34.429328+0000","last_undegraded":"2023-07-29T14:05:34.429328+0000","last_fullsized":"2023-07-29T14:05:34.429328+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:55:19.929908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1003,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.046757+0000","last_change":"2023-07-29T13:56:55.572081+0000","last_active":"2023-07-29T14:05:37.046757+0000","last_peered":"2023-07-29T14:05:37.046757+0000","last_clean":"2023-07-29T14:05:37.046757+0000","last_became_active":"2023-07-29T13:56:55.564155+0000","last_became_peered":"2023-07-29T13:56:55.564155+0000","last_unstale":"2023-07-29T14:05:37.046757+0000","last_undegraded":"2023-07-29T14:05:37.046757+0000","last_fullsized":"2023-07-29T14:05:37.046757+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:56:05.370398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":971,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.554873+0000","last_change":"2023-07-29T13:53:19.441193+0000","last_active":"2023-07-29T14:05:35.554873+0000","last_peered":"2023-07-29T14:05:35.554873+0000","last_clean":"2023-07-29T14:05:35.554873+0000","last_became_active":"2023-07-29T13:53:19.433950+0000","last_became_peered":"2023-07-29T13:53:19.433950+0000","last_unstale":"2023-07-29T14:05:35.554873+0000","last_undegraded":"2023-07-29T14:05:35.554873+0000","last_fullsized":"2023-07-29T14:05:35.554873+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:38:39.350563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":950,"reported_epoch":1002,"state":"peering","last_fresh":"2023-07-29T14:05:36.299801+0000","last_change":"2023-07-29T14:05:36.303408+0000","last_active":"2023-07-29T14:05:36.299801+0000","last_peered":"2023-07-29T14:05:35.502600+0000","last_clean":"2023-07-29T14:04:06.286440+0000","last_became_active":"2023-07-29T14:05:29.629683+0000","last_became_peered":"2023-07-29T14:05:29.629683+0000","last_unstale":"2023-07-29T14:05:36.299801+0000","last_undegraded":"2023-07-29T14:05:36.299801+0000","last_fullsized":"2023-07-29T14:05:36.299801+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:52:32.253454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":17,"seq":73014444472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60040,"kb_used_data":6848,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":7012352,"data_stored":3836586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":70,"apply_latency_ms":70,"commit_latency_ns":70000000,"apply_latency_ns":70000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":15.343999999999999,"5min":16.634,"15min":170.43700000000001},"min":{"1min":6.3490000000000002,"5min":6.3330000000000002,"15min":4.7590000000000003},"max":{"1min":111.66,"5min":172.44399999999999,"15min":12751.485000000001},"last":8.1539999999999999},{"interface":"front","average":{"1min":16.254000000000001,"5min":16.774000000000001,"15min":173.22},"min":{"1min":6.79,"5min":5.4500000000000002,"15min":5.4500000000000002},"max":{"1min":129.28399999999999,"5min":173.48699999999999,"15min":12747.76},"last":15.557}]},{"osd":1,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":18.536000000000001,"5min":1543.261,"15min":525.65800000000002},"min":{"1min":7.0010000000000003,"5min":6.5149999999999997,"15min":6.5149999999999997},"max":{"1min":99.557000000000002,"5min":29038.089,"15min":29038.089},"last":58.942},{"interface":"front","average":{"1min":22.518000000000001,"5min":1548.174,"15min":530.21500000000003},"min":{"1min":6.9089999999999998,"5min":6.4749999999999996,"15min":6.2249999999999996},"max":{"1min":102.327,"5min":29039.032999999999,"15min":29039.032999999999},"last":53.643999999999998}]},{"osd":2,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":23.297000000000001,"5min":21.439,"15min":20.452000000000002},"min":{"1min":7.992,"5min":7.3200000000000003,"15min":7.3200000000000003},"max":{"1min":104.566,"5min":249.84800000000001,"15min":784.66200000000003},"last":21.434000000000001},{"interface":"front","average":{"1min":21.670000000000002,"5min":21.084,"15min":19.474},"min":{"1min":8.6280000000000001,"5min":8.3919999999999995,"15min":7.8049999999999997},"max":{"1min":99.706000000000003,"5min":202.64500000000001,"15min":368.62200000000001},"last":25.123999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.91,"5min":25.608000000000001,"15min":478.83800000000002},"min":{"1min":7.0910000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"max":{"1min":101.714,"5min":312.73599999999999,"15min":28381.707999999999},"last":22.844000000000001},{"interface":"front","average":{"1min":20.298999999999999,"5min":27.803999999999998,"15min":474.06299999999999},"min":{"1min":9.3219999999999992,"5min":9.3219999999999992,"15min":8.7520000000000007},"max":{"1min":97.727999999999994,"5min":242.63,"15min":28126.012999999999},"last":25.978000000000002}]},{"osd":5,"last update":"Sat Jul 29 14:05:01 2023","interfaces":[{"interface":"back","average":{"1min":17.222999999999999,"5min":20.141999999999999,"15min":32.993000000000002},"min":{"1min":10.452,"5min":8.4540000000000006,"15min":7.3689999999999998},"max":{"1min":79.867000000000004,"5min":85.578999999999994,"15min":488.72300000000001},"last":58.226999999999997},{"interface":"front","average":{"1min":14.161,"5min":19.236999999999998,"15min":21.838000000000001},"min":{"1min":8.1379999999999999,"5min":8.1379999999999999,"15min":7.3840000000000003},"max":{"1min":52.622,"5min":106.634,"15min":128.791},"last":51.494}]},{"osd":6,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":24.904,"5min":20.061,"15min":22.629999999999999},"min":{"1min":9.1470000000000002,"5min":8.0129999999999999,"15min":7.7779999999999996},"max":{"1min":106.33199999999999,"5min":124.819,"15min":787.18499999999995},"last":54.243000000000002},{"interface":"front","average":{"1min":25.198,"5min":20.042000000000002,"15min":22.454999999999998},"min":{"1min":9.8010000000000002,"5min":8.5790000000000006,"15min":8.3170000000000002},"max":{"1min":112.328,"5min":112.328,"15min":646.87599999999998},"last":35.186999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":27.215,"5min":22.292000000000002,"15min":48.816000000000003},"min":{"1min":11.487,"5min":9.2620000000000005,"15min":9.2620000000000005},"max":{"1min":149.43199999999999,"5min":149.43199999999999,"15min":648.54200000000003},"last":36.865000000000002},{"interface":"front","average":{"1min":23.321000000000002,"5min":20.568999999999999,"15min":38.241999999999997},"min":{"1min":10.385,"5min":9.2300000000000004,"15min":8.7309999999999999},"max":{"1min":75.954999999999998,"5min":75.954999999999998,"15min":364.86500000000001},"last":30.257999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444484,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64112,"kb_used_data":6824,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94307728,"statfs":{"total":96636764160,"available":96571113472,"internally_reserved":0,"allocated":6987776,"data_stored":3804439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":13.247999999999999,"15min":130.82599999999999},"min":{"1min":5.3789999999999996,"5min":5.0209999999999999,"15min":4.476},"max":{"1min":29.634,"5min":93.590999999999994,"15min":10009.493},"last":12.676},{"interface":"front","average":{"1min":13.936999999999999,"5min":13.526,"15min":131.76599999999999},"min":{"1min":7.1299999999999999,"5min":6.2069999999999999,"15min":5.617},"max":{"1min":37.527999999999999,"5min":99.156999999999996,"15min":9995.5869999999995},"last":10.468}]},{"osd":1,"last update":"Sat Jul 29 14:04:47 2023","interfaces":[{"interface":"back","average":{"1min":11.255000000000001,"5min":1413.5239999999999,"15min":481.31099999999998},"min":{"1min":6.0899999999999999,"5min":6.0899999999999999,"15min":6.0899999999999999},"max":{"1min":17.850999999999999,"5min":29343.018,"15min":29343.018},"last":101.836},{"interface":"front","average":{"1min":11.234999999999999,"5min":1411.289,"15min":478.77199999999999},"min":{"1min":7.0209999999999999,"5min":6.5880000000000001,"15min":5.8860000000000001},"max":{"1min":18.347000000000001,"5min":29342.263999999999,"15min":29342.263999999999},"last":137.94}]},{"osd":3,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":17.623000000000001,"5min":20.777000000000001,"15min":365.04399999999998},"min":{"1min":6.7809999999999997,"5min":6.7809999999999997,"15min":6.7809999999999997},"max":{"1min":86.926000000000002,"5min":331.92399999999998,"15min":29770.313999999998},"last":11.025},{"interface":"front","average":{"1min":21.684999999999999,"5min":22.039999999999999,"15min":358.54500000000002},"min":{"1min":9.3070000000000004,"5min":8.0540000000000003,"15min":6.4749999999999996},"max":{"1min":89.156000000000006,"5min":332.49900000000002,"15min":29586.387999999999},"last":7.351}]},{"osd":4,"last update":"Sat Jul 29 14:04:53 2023","interfaces":[{"interface":"back","average":{"1min":12.471,"5min":819.34000000000003,"15min":870.13199999999995},"min":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"max":{"1min":52.584000000000003,"5min":25685.629000000001,"15min":32071.659},"last":6.5810000000000004},{"interface":"front","average":{"1min":15.125999999999999,"5min":814.09699999999998,"15min":870.976},"min":{"1min":8.8330000000000002,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":35.020000000000003,"5min":25610.863000000001,"15min":32126.150000000001},"last":8.0410000000000004}]},{"osd":5,"last update":"Sat Jul 29 14:05:06 2023","interfaces":[{"interface":"back","average":{"1min":14.398,"5min":20.428999999999998,"15min":38.106999999999999},"min":{"1min":8.7579999999999991,"5min":8.4269999999999996,"15min":8.4269999999999996},"max":{"1min":18.667000000000002,"5min":193.45400000000001,"15min":428.04000000000002},"last":11.632999999999999},{"interface":"front","average":{"1min":12.019,"5min":20.446999999999999,"15min":41.822000000000003},"min":{"1min":6.891,"5min":6.891,"15min":6.891},"max":{"1min":19.634,"5min":174.292,"15min":430.15499999999997},"last":10.163}]},{"osd":6,"last update":"Sat Jul 29 14:04:38 2023","interfaces":[{"interface":"back","average":{"1min":14.581,"5min":23.431000000000001,"15min":19.853000000000002},"min":{"1min":8.5359999999999996,"5min":8.5359999999999996,"15min":8.5359999999999996},"max":{"1min":24.651,"5min":318.45299999999997,"15min":318.45299999999997},"last":12.380000000000001},{"interface":"front","average":{"1min":15.903,"5min":21.638000000000002,"15min":20.547999999999998},"min":{"1min":11.417999999999999,"5min":9.7789999999999999,"15min":7.5300000000000002},"max":{"1min":22.896999999999998,"5min":349.34899999999999,"15min":349.34899999999999},"last":9.2230000000000008}]},{"osd":7,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":15.279999999999999,"5min":20.536000000000001,"15min":21.027000000000001},"min":{"1min":10.663,"5min":8.5079999999999991,"15min":7.5890000000000004},"max":{"1min":20.757999999999999,"5min":291.80900000000003,"15min":291.80900000000003},"last":25.786000000000001},{"interface":"front","average":{"1min":15.644,"5min":21.513000000000002,"15min":22.518000000000001},"min":{"1min":9.6419999999999995,"5min":9.1940000000000008,"15min":9.1940000000000008},"max":{"1min":23.408000000000001,"5min":291.44200000000001,"15min":291.44200000000001},"last":82.373999999999995}]}]},{"osd":1,"up_from":447,"seq":1919850381742,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60408,"kb_used_data":7216,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311432,"statfs":{"total":96636764160,"available":96574906368,"internally_reserved":0,"allocated":7389184,"data_stored":4202279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":20.931000000000001,"5min":20.015999999999998,"15min":112.807},"min":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.3680000000000003},"max":{"1min":116.512,"5min":364.47800000000001,"15min":12525.57},"last":10.975},{"interface":"front","average":{"1min":18.943000000000001,"5min":20.664000000000001,"15min":112.874},"min":{"1min":6.0970000000000004,"5min":6.0970000000000004,"15min":4.6820000000000004},"max":{"1min":100.402,"5min":372.00999999999999,"15min":12523.974},"last":8.8849999999999998}]},{"osd":2,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.987,"5min":26.809000000000001,"15min":19.399999999999999},"min":{"1min":7.819,"5min":6.5110000000000001,"15min":5.5919999999999996},"max":{"1min":69.497,"5min":360.88099999999997,"15min":360.88099999999997},"last":8.4369999999999994},{"interface":"front","average":{"1min":19.43,"5min":28.155999999999999,"15min":20.445},"min":{"1min":7.093,"5min":7.093,"15min":6.2729999999999997},"max":{"1min":93.792000000000002,"5min":368.51400000000001,"15min":368.51400000000001},"last":7.2009999999999996}]},{"osd":3,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":26.696000000000002,"5min":30.559000000000001,"15min":531.69799999999998},"min":{"1min":9.1590000000000007,"5min":6.7140000000000004,"15min":6.7140000000000004},"max":{"1min":216.16300000000001,"5min":435.92200000000003,"15min":34504.942000000003},"last":18.053000000000001},{"interface":"front","average":{"1min":31.332000000000001,"5min":41.100999999999999,"15min":539.97299999999996},"min":{"1min":10.423,"5min":10.423,"15min":5.3639999999999999},"max":{"1min":238.33699999999999,"5min":700.10500000000002,"15min":34620.970999999998},"last":29.898}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.427,"5min":1170.991,"15min":980.98599999999999},"min":{"1min":8.4130000000000003,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":50.289000000000001,"5min":25247.098999999998,"15min":32258.529999999999},"last":11.036},{"interface":"front","average":{"1min":16.536000000000001,"5min":1165.587,"15min":967.39999999999998},"min":{"1min":11.340999999999999,"5min":8.9169999999999998,"15min":6.5090000000000003},"max":{"1min":29.376999999999999,"5min":25174.008999999998,"15min":31933.992999999999},"last":9.3900000000000006}]},{"osd":5,"last update":"Sat Jul 29 14:05:13 2023","interfaces":[{"interface":"back","average":{"1min":15.249000000000001,"5min":33.526000000000003,"15min":43.616},"min":{"1min":10.416,"5min":10.294,"15min":10.294},"max":{"1min":23.649999999999999,"5min":433.79300000000001,"15min":433.79300000000001},"last":81.477999999999994},{"interface":"front","average":{"1min":15.952,"5min":36.515000000000001,"15min":46.286000000000001},"min":{"1min":11.467000000000001,"5min":9.6929999999999996,"15min":8.7260000000000009},"max":{"1min":22.300999999999998,"5min":419.22800000000001,"15min":419.22800000000001},"last":88.980999999999995}]},{"osd":6,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":16.931999999999999,"5min":21.992000000000001,"15min":19.459},"min":{"1min":6.6079999999999997,"5min":6.327,"15min":6.327},"max":{"1min":154.92400000000001,"5min":357.20499999999998,"15min":357.20499999999998},"last":15.124000000000001},{"interface":"front","average":{"1min":20.251999999999999,"5min":27.460999999999999,"15min":23.411000000000001},"min":{"1min":10.877000000000001,"5min":9.5079999999999991,"15min":9.5079999999999991},"max":{"1min":155.994,"5min":359.68099999999998,"15min":359.68099999999998},"last":14.048}]},{"osd":7,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.683,"5min":35.137999999999998,"15min":23.725999999999999},"min":{"1min":10.130000000000001,"5min":8.5199999999999996,"15min":7.5919999999999996},"max":{"1min":92.168999999999997,"5min":563.29700000000003,"15min":563.29700000000003},"last":12.036},{"interface":"front","average":{"1min":17.949000000000002,"5min":30.442,"15min":21.349},"min":{"1min":8.2240000000000002,"5min":8.2240000000000002,"15min":7.2919999999999998},"max":{"1min":113.15900000000001,"5min":352.65199999999999,"15min":352.65199999999999},"last":9.9580000000000002}]}]},{"osd":0,"up_from":1002,"seq":4303557231028,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60024,"kb_used_data":6472,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311816,"statfs":{"total":96636764160,"available":96575299584,"internally_reserved":0,"allocated":6627328,"data_stored":3587977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":77,"apply_latency_ms":77,"commit_latency_ns":77000000,"apply_latency_ns":77000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":71.099999999999994,"5min":1023.201,"15min":351.10899999999998},"min":{"1min":8.9689999999999994,"5min":5.9249999999999998,"15min":4.6269999999999998},"max":{"1min":350.00200000000001,"5min":27534,"15min":27534},"last":6.1740000000000004},{"interface":"front","average":{"1min":99.875,"5min":1025.7080000000001,"15min":350.98899999999998},"min":{"1min":5.9589999999999996,"5min":5.7240000000000002,"15min":5.0739999999999998},"max":{"1min":591.69600000000003,"5min":27535.008999999998,"15min":27535.008999999998},"last":6.6829999999999998}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.170999999999999,"5min":22.285,"15min":18.638000000000002},"min":{"1min":11.680999999999999,"5min":8.1069999999999993,"15min":6.6159999999999997},"max":{"1min":364.62799999999999,"5min":364.62799999999999,"15min":364.62799999999999},"last":9.9009999999999998},{"interface":"front","average":{"1min":45.890000000000001,"5min":26.417999999999999,"15min":21.443000000000001},"min":{"1min":14.172000000000001,"5min":10.023999999999999,"15min":8.8780000000000001},"max":{"1min":370.34300000000002,"5min":370.34300000000002,"15min":370.34300000000002},"last":11.736000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":66.832999999999998,"5min":25.178000000000001,"15min":355.89100000000002},"min":{"1min":6.7610000000000001,"5min":6.4409999999999998,"15min":5.891},"max":{"1min":444.59500000000003,"5min":444.59500000000003,"15min":30170.040000000001},"last":96.820999999999998},{"interface":"front","average":{"1min":65.754000000000005,"5min":33.606000000000002,"15min":363.07400000000001},"min":{"1min":10.266999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"max":{"1min":386.06099999999998,"5min":386.06099999999998,"15min":30171.194},"last":74.893000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":92.555000000000007,"5min":36.014000000000003,"15min":585.43899999999996},"min":{"1min":12.201000000000001,"5min":6.4480000000000004,"15min":6.0979999999999999},"max":{"1min":445.65300000000002,"5min":445.65300000000002,"15min":29892.669999999998},"last":11.266},{"interface":"front","average":{"1min":115.806,"5min":40.560000000000002,"15min":588.88900000000001},"min":{"1min":7.923,"5min":5.9400000000000004,"15min":5.9400000000000004},"max":{"1min":593.13499999999999,"5min":593.13499999999999,"15min":29877.289000000001},"last":14.489000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.341000000000001,"5min":28.972000000000001,"15min":56.932000000000002},"min":{"1min":12.702,"5min":9.5700000000000003,"15min":9.5700000000000003},"max":{"1min":403.13999999999999,"5min":403.13999999999999,"15min":508.02100000000002},"last":94.206999999999994},{"interface":"front","average":{"1min":43.966999999999999,"5min":23.216000000000001,"15min":32.374000000000002},"min":{"1min":7.6429999999999998,"5min":7.1210000000000004,"15min":7.1210000000000004},"max":{"1min":590.58500000000004,"5min":590.58500000000004,"15min":590.58500000000004},"last":92.817999999999998}]},{"osd":6,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":52.365000000000002,"5min":30.853999999999999,"15min":25.227},"min":{"1min":16.876000000000001,"5min":10.409000000000001,"15min":9.9350000000000005},"max":{"1min":375.36500000000001,"5min":375.36500000000001,"15min":375.36500000000001},"last":51.311},{"interface":"front","average":{"1min":55.634,"5min":32.579999999999998,"15min":27.417999999999999},"min":{"1min":18.984999999999999,"5min":11.564,"15min":11.564},"max":{"1min":380.31599999999997,"5min":380.31599999999997,"15min":380.31599999999997},"last":71.132999999999996}]},{"osd":7,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":57.960999999999999,"5min":28.638000000000002,"15min":20.780000000000001},"min":{"1min":17.748999999999999,"5min":8.4939999999999998,"15min":7.9349999999999996},"max":{"1min":385.01900000000001,"5min":385.01900000000001,"15min":385.01900000000001},"last":13.416},{"interface":"front","average":{"1min":54.853999999999999,"5min":29.765999999999998,"15min":25.545000000000002},"min":{"1min":10.992000000000001,"5min":8.1229999999999993,"15min":8.1229999999999993},"max":{"1min":446.69099999999997,"5min":446.69099999999997,"15min":446.69099999999997},"last":15.956}]}]},{"osd":4,"up_from":16,"seq":68719477161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60104,"kb_used_data":6912,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311736,"statfs":{"total":96636764160,"available":96575217664,"internally_reserved":0,"allocated":7077888,"data_stored":3907142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":13.432,"5min":21.457000000000001,"15min":87.253},"min":{"1min":6.4029999999999996,"5min":4.6230000000000002,"15min":4.6230000000000002},"max":{"1min":48.048999999999999,"5min":495.86900000000003,"15min":10627.831},"last":11.238},{"interface":"front","average":{"1min":18.010000000000002,"5min":25.013000000000002,"15min":90.852000000000004},"min":{"1min":7.2400000000000002,"5min":6.0990000000000002,"15min":5.6660000000000004},"max":{"1min":124.64100000000001,"5min":446.84100000000001,"15min":10653.877},"last":7.7329999999999997}]},{"osd":1,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":11.276999999999999,"5min":1441.567,"15min":495.98200000000003},"min":{"1min":7.2640000000000002,"5min":7.2640000000000002,"15min":5.9909999999999997},"max":{"1min":21.369,"5min":26872.857,"15min":26872.857},"last":8.4139999999999997},{"interface":"front","average":{"1min":11.013,"5min":1440.0119999999999,"15min":493.40499999999997},"min":{"1min":7.4269999999999996,"5min":7.3090000000000002,"15min":5.6909999999999998},"max":{"1min":17.440999999999999,"5min":26874.048999999999,"15min":26874.048999999999},"last":10.928000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":12.445,"5min":24.050999999999998,"15min":23.050000000000001},"min":{"1min":8.9000000000000004,"5min":7.3019999999999996,"15min":7.3019999999999996},"max":{"1min":15.904999999999999,"5min":453.40100000000001,"15min":597.11800000000005},"last":17.305},{"interface":"front","average":{"1min":15.029999999999999,"5min":27.077999999999999,"15min":26.128},"min":{"1min":12.429,"5min":11.631,"15min":8.0559999999999992},"max":{"1min":19.111000000000001,"5min":453.96199999999999,"15min":705.04499999999996},"last":57.628}]},{"osd":3,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":14.789999999999999,"5min":27.626999999999999,"15min":590.51199999999994},"min":{"1min":9.7219999999999995,"5min":7.6799999999999997,"15min":5.9740000000000002},"max":{"1min":23.942,"5min":448.65100000000001,"15min":30725.612000000001},"last":13.273999999999999},{"interface":"front","average":{"1min":12.497,"5min":20.047999999999998,"15min":582.06899999999996},"min":{"1min":7.7690000000000001,"5min":6.3840000000000003,"15min":6.3840000000000003},"max":{"1min":19.815000000000001,"5min":336.62599999999998,"15min":30407.659},"last":15.597}]},{"osd":5,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":16.190999999999999,"5min":30.579000000000001,"15min":32.869},"min":{"1min":14.1,"5min":7.1440000000000001,"15min":7.1440000000000001},"max":{"1min":18.738,"5min":450.81700000000001,"15min":450.81700000000001},"last":14.425000000000001},{"interface":"front","average":{"1min":10.430999999999999,"5min":19.959,"15min":22.803999999999998},"min":{"1min":6.2779999999999996,"5min":6.2779999999999996,"15min":6.2779999999999996},"max":{"1min":20.254999999999999,"5min":340.04000000000002,"15min":340.04000000000002},"last":7.9279999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":13.007,"5min":22.312000000000001,"15min":22.866},"min":{"1min":8.1120000000000001,"5min":7.1689999999999996,"15min":6.1520000000000001},"max":{"1min":22.010000000000002,"5min":339.47199999999998,"15min":696.952},"last":10.199999999999999},{"interface":"front","average":{"1min":14.724,"5min":23.917999999999999,"15min":23.209},"min":{"1min":9.0289999999999999,"5min":6.3449999999999998,"15min":6.3449999999999998},"max":{"1min":20.968,"5min":498.26999999999998,"15min":598.86900000000003},"last":16.390999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":12.805,"5min":24.097000000000001,"15min":24.802},"min":{"1min":8.7430000000000003,"5min":8.3399999999999999,"15min":7.7930000000000001},"max":{"1min":18.638000000000002,"5min":340.61900000000003,"15min":699.31799999999998},"last":13.837999999999999},{"interface":"front","average":{"1min":17.483000000000001,"5min":31.318999999999999,"15min":33.040999999999997},"min":{"1min":13.861000000000001,"5min":11.067,"15min":11.067},"max":{"1min":19.817,"5min":455.339,"15min":703.68200000000002},"last":11.691000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82276,"kb_used_data":6940,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94289564,"statfs":{"total":96636764160,"available":96552513536,"internally_reserved":0,"allocated":7106560,"data_stored":3935814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":12.891999999999999,"5min":16.984000000000002,"15min":62.423000000000002},"min":{"1min":6.9390000000000001,"5min":5.2350000000000003,"15min":5.2350000000000003},"max":{"1min":29.821000000000002,"5min":224.93899999999999,"15min":9502.5869999999995},"last":41.209000000000003},{"interface":"front","average":{"1min":15.925000000000001,"5min":19.867000000000001,"15min":65.921999999999997},"min":{"1min":8.2050000000000001,"5min":6.1580000000000004,"15min":6.1580000000000004},"max":{"1min":31.303999999999998,"5min":130.48400000000001,"15min":9511.3719999999994},"last":64.659000000000006}]},{"osd":1,"last update":"Sat Jul 29 14:04:49 2023","interfaces":[{"interface":"back","average":{"1min":17.140999999999998,"5min":970.20899999999995,"15min":335.46300000000002},"min":{"1min":7.5839999999999996,"5min":7.3369999999999997,"15min":4.8529999999999998},"max":{"1min":73.679000000000002,"5min":26903.921999999999,"15min":26903.921999999999},"last":8.5850000000000009},{"interface":"front","average":{"1min":12.878,"5min":963.24699999999996,"15min":332.12599999999998},"min":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.3330000000000002},"max":{"1min":69.432000000000002,"5min":26893.022000000001,"15min":26893.022000000001},"last":5.3620000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":14.891999999999999,"5min":21.295999999999999,"15min":19.027000000000001},"min":{"1min":10.173,"5min":7.8609999999999998,"15min":6.6299999999999999},"max":{"1min":26.302,"5min":191.73099999999999,"15min":191.73099999999999},"last":26.302},{"interface":"front","average":{"1min":11.848000000000001,"5min":16.003,"15min":15.422000000000001},"min":{"1min":7.4690000000000003,"5min":5.5499999999999998,"15min":5.5499999999999998},"max":{"1min":18,"5min":189.40199999999999,"15min":189.40199999999999},"last":11.739000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":10.653,"5min":15.525,"15min":521.48800000000006},"min":{"1min":7.2619999999999996,"5min":6.335,"15min":6.335},"max":{"1min":23.292000000000002,"5min":187.852,"15min":30710.619999999999},"last":37.712000000000003},{"interface":"front","average":{"1min":15.202999999999999,"5min":17.899000000000001,"15min":521.02300000000002},"min":{"1min":9.907,"5min":7.7009999999999996,"15min":7.6180000000000003},"max":{"1min":40.335000000000001,"5min":101.88200000000001,"15min":30591.776999999998},"last":38.539999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:56 2023","interfaces":[{"interface":"back","average":{"1min":16.030000000000001,"5min":808.61300000000006,"15min":1019.547},"min":{"1min":8.0950000000000006,"5min":6.718,"15min":6.1070000000000002},"max":{"1min":73.055999999999997,"5min":25620.438999999998,"15min":32899.582000000002},"last":8.2870000000000008},{"interface":"front","average":{"1min":16.128,"5min":828.35799999999995,"15min":1009.853},"min":{"1min":10.254,"5min":6.2089999999999996,"15min":6.2089999999999996},"max":{"1min":40.786999999999999,"5min":25979.712,"15min":32485.278999999999},"last":9.7769999999999992}]},{"osd":5,"last update":"Sat Jul 29 14:05:04 2023","interfaces":[{"interface":"back","average":{"1min":17.489999999999998,"5min":21.376999999999999,"15min":24.847000000000001},"min":{"1min":11.52,"5min":9.4949999999999992,"15min":8.9290000000000003},"max":{"1min":34.677,"5min":75.25,"15min":189.423},"last":94.355000000000004},{"interface":"front","average":{"1min":14.324,"5min":18.073,"15min":21.690000000000001},"min":{"1min":8.6969999999999992,"5min":7.3330000000000002,"15min":7.3330000000000002},"max":{"1min":24.77,"5min":80.085999999999999,"15min":188.518},"last":91.700999999999993}]},{"osd":7,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":24.321000000000002,"5min":19.489999999999998,"15min":19.814},"min":{"1min":14.164,"5min":7.1059999999999999,"15min":7.1059999999999999},"max":{"1min":74.337000000000003,"5min":74.337000000000003,"15min":200.55099999999999},"last":13.702999999999999},{"interface":"front","average":{"1min":18.581,"5min":20.780000000000001,"15min":22.265000000000001},"min":{"1min":9.6880000000000006,"5min":8.0790000000000006,"15min":8.0790000000000006},"max":{"1min":47.313000000000002,"5min":89.125,"15min":222.65199999999999},"last":12.701000000000001}]}]},{"osd":5,"up_from":704,"seq":3023656976772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78444,"kb_used_data":7204,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94293396,"statfs":{"total":96636764160,"available":96556437504,"internally_reserved":0,"allocated":7376896,"data_stored":4202582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":78,"apply_latency_ms":78,"commit_latency_ns":78000000,"apply_latency_ns":78000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:04:00 2023","interfaces":[{"interface":"back","average":{"1min":23.48,"5min":22.623000000000001,"15min":41.000999999999998},"min":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":4.742},"max":{"1min":150.63,"5min":262.56099999999998,"15min":299.87799999999999},"last":9.6039999999999992},{"interface":"front","average":{"1min":24.837,"5min":22.727,"15min":52.545999999999999},"min":{"1min":5.8259999999999996,"5min":4.6539999999999999,"15min":4.6539999999999999},"max":{"1min":111.14100000000001,"5min":253.80000000000001,"15min":370.00999999999999},"last":7.3899999999999997}]},{"osd":1,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":13.867000000000001,"5min":25.277000000000001,"15min":33.274999999999999},"min":{"1min":6.5419999999999998,"5min":5.3780000000000001,"15min":5.3780000000000001},"max":{"1min":91.590000000000003,"5min":362.58800000000002,"15min":362.58800000000002},"last":7.0529999999999999},{"interface":"front","average":{"1min":15.465999999999999,"5min":34.079000000000001,"15min":56.872999999999998},"min":{"1min":7.5650000000000004,"5min":6.8680000000000003,"15min":6.8680000000000003},"max":{"1min":85.760000000000005,"5min":356.69900000000001,"15min":363.50999999999999},"last":9.3510000000000009}]},{"osd":2,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.152000000000001,"5min":26.193000000000001,"15min":51.703000000000003},"min":{"1min":8.1880000000000006,"5min":7.032,"15min":7.032},"max":{"1min":151.804,"5min":312.755,"15min":384.024},"last":6.0259999999999998},{"interface":"front","average":{"1min":16.754000000000001,"5min":20.221,"15min":25.251000000000001},"min":{"1min":7.782,"5min":7.2759999999999998,"15min":7.2709999999999999},"max":{"1min":102.09099999999999,"5min":228.55600000000001,"15min":228.55600000000001},"last":11.063000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":28.802,"15min":55.957999999999998},"min":{"1min":10.734,"5min":7.4100000000000001,"15min":7.4100000000000001},"max":{"1min":131.36600000000001,"5min":312.12200000000001,"15min":381.23200000000003},"last":12.308},{"interface":"front","average":{"1min":18.568999999999999,"5min":28.875,"15min":66.894999999999996},"min":{"1min":7.944,"5min":7.944,"15min":7.944},"max":{"1min":118.456,"5min":265.92700000000002,"15min":382.779},"last":24.484999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":19.385999999999999,"5min":1108.5630000000001,"15min":412.19600000000003},"min":{"1min":10.5,"5min":7.343,"15min":7.3070000000000004},"max":{"1min":103.768,"5min":23572.741999999998,"15min":23572.741999999998},"last":8.423},{"interface":"front","average":{"1min":20.039000000000001,"5min":1113.885,"15min":428.73500000000001},"min":{"1min":8.7260000000000009,"5min":6.8929999999999998,"15min":6.8929999999999998},"max":{"1min":137.57400000000001,"5min":23610.142,"15min":23610.142},"last":10.224}]},{"osd":6,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":17.074000000000002,"5min":25.704999999999998,"15min":51.972000000000001},"min":{"1min":9.7360000000000007,"5min":8.1709999999999994,"15min":8.1709999999999994},"max":{"1min":41.347000000000001,"5min":263.983,"15min":450.17500000000001},"last":25.594000000000001},{"interface":"front","average":{"1min":18.436,"5min":26.684999999999999,"15min":57.575000000000003},"min":{"1min":11.185,"5min":8.1969999999999992,"15min":8.1969999999999992},"max":{"1min":63.707999999999998,"5min":221.99199999999999,"15min":417.303},"last":14.269}]},{"osd":7,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":21.550999999999998,"5min":28.064,"15min":47.530999999999999},"min":{"1min":12.340999999999999,"5min":11.222,"15min":10.602},"max":{"1min":104.718,"5min":284.48000000000002,"15min":341.30500000000001},"last":12.962999999999999},{"interface":"front","average":{"1min":18.460999999999999,"5min":27.806000000000001,"15min":53.993000000000002},"min":{"1min":12.089,"5min":11.042999999999999,"15min":11.042999999999999},"max":{"1min":46.353999999999999,"5min":191.33099999999999,"15min":329.86500000000001},"last":28.317}]}]},{"osd":7,"up_from":650,"seq":2791728742800,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60024,"kb_used_data":6472,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311816,"statfs":{"total":96636764160,"available":96575299584,"internally_reserved":0,"allocated":6627328,"data_stored":3587977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:45 2023","interfaces":[{"interface":"back","average":{"1min":11.94,"5min":13.635,"15min":23.015000000000001},"min":{"1min":5.5890000000000004,"5min":5.4539999999999997,"15min":5.4539999999999997},"max":{"1min":28.5,"5min":75.968000000000004,"15min":293.94499999999999},"last":11.478999999999999},{"interface":"front","average":{"1min":14.968999999999999,"5min":18.658999999999999,"15min":28.620000000000001},"min":{"1min":6.8179999999999996,"5min":6.8179999999999996,"15min":6.8179999999999996},"max":{"1min":29.143999999999998,"5min":116.952,"15min":358.267},"last":7.9210000000000003}]},{"osd":1,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":19.048999999999999,"5min":1108.855,"15min":389.67099999999999},"min":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":4.6660000000000004},"max":{"1min":184.25800000000001,"5min":27546.963,"15min":27546.963},"last":12.223000000000001},{"interface":"front","average":{"1min":22.969999999999999,"5min":1111.8620000000001,"15min":385.10300000000001},"min":{"1min":8.6110000000000007,"5min":5.5010000000000003,"15min":5.5010000000000003},"max":{"1min":150.75700000000001,"5min":27546.280999999999,"15min":27546.280999999999},"last":14.646000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.088000000000001,"5min":19.381,"15min":28.02},"min":{"1min":10.154999999999999,"5min":7.9100000000000001,"15min":7.9100000000000001},"max":{"1min":103.09,"5min":103.09,"15min":461.83800000000002},"last":12.978999999999999},{"interface":"front","average":{"1min":21.068000000000001,"5min":15.704000000000001,"15min":23.861999999999998},"min":{"1min":7.1269999999999998,"5min":6.6239999999999997,"15min":6.6239999999999997},"max":{"1min":181.97900000000001,"5min":181.97900000000001,"15min":385.40800000000002},"last":8.9149999999999991}]},{"osd":3,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":25.065999999999999,"5min":21.029,"15min":1531.4280000000001},"min":{"1min":7.976,"5min":7.6660000000000004,"15min":7.6660000000000004},"max":{"1min":242.197,"5min":242.197,"15min":34550.838000000003},"last":16.02},{"interface":"front","average":{"1min":31.428999999999998,"5min":27.873000000000001,"15min":1543.2929999999999},"min":{"1min":12.592000000000001,"5min":6.9550000000000001,"15min":6.9550000000000001},"max":{"1min":240.46899999999999,"5min":243.06399999999999,"15min":34556.381000000001},"last":18.369}]},{"osd":4,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":20.402999999999999,"5min":1074.817,"15min":900.84500000000003},"min":{"1min":6.7220000000000004,"5min":6.3719999999999999,"15min":6.3719999999999999},"max":{"1min":103.931,"5min":25146.41,"15min":27891.68},"last":9.7100000000000009},{"interface":"front","average":{"1min":29.045999999999999,"5min":1099.6300000000001,"15min":918.40200000000004},"min":{"1min":9.7360000000000007,"5min":8.1630000000000003,"15min":7.2699999999999996},"max":{"1min":189.47900000000001,"5min":25434.151999999998,"15min":27883.419999999998},"last":21.300999999999998}]},{"osd":5,"last update":"Sat Jul 29 14:05:08 2023","interfaces":[{"interface":"back","average":{"1min":17.529,"5min":28.314,"15min":72.040999999999997},"min":{"1min":14.367000000000001,"5min":11.215,"15min":10.683},"max":{"1min":22.012,"5min":230.16800000000001,"15min":1265.123},"last":21.904},{"interface":"front","average":{"1min":12.343999999999999,"5min":25.687999999999999,"15min":76.659000000000006},"min":{"1min":9.609,"5min":7.71,"15min":7.5540000000000003},"max":{"1min":21.030999999999999,"5min":255.23699999999999,"15min":1234.9300000000001},"last":15.369999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":21.149999999999999,"15min":25.884},"min":{"1min":10.222,"5min":9.9740000000000002,"15min":7.8689999999999998},"max":{"1min":78.843000000000004,"5min":95.558999999999997,"15min":243.66499999999999},"last":16.655000000000001},{"interface":"front","average":{"1min":24.378,"5min":25.236999999999998,"15min":32.558},"min":{"1min":12.111000000000001,"5min":10.542,"15min":10.542},"max":{"1min":92.266999999999996,"5min":148.334,"15min":454.68799999999999},"last":19.009}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-29T14:05:40.793 INFO:tasks.ceph.ceph_manager.ceph:PG 7.11 is not active+clean 2023-07-29T14:05:40.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 962, 'reported_epoch': 1003, 'state': 'activating', 'last_fresh': '2023-07-29T14:05:37.507347+0000', 'last_change': '2023-07-29T14:05:37.515837+0000', 'last_active': '2023-07-29T14:05:36.327038+0000', 'last_peered': '2023-07-29T14:05:35.596454+0000', 'last_clean': '2023-07-29T14:04:22.472067+0000', 'last_became_active': '2023-07-29T14:05:29.733569+0000', 'last_became_peered': '2023-07-29T14:05:29.733569+0000', 'last_unstale': '2023-07-29T14:05:37.507347+0000', 'last_undegraded': '2023-07-29T14:05:37.507347+0000', 'last_fullsized': '2023-07-29T14:05:37.507347+0000', 'mapping_epoch': 1002, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 26, 'last_epoch_clean': 45, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-29T13:29:15.987740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-29T13:29:15.987740+0000', 'last_clean_scrub_stamp': '2023-07-29T13:29:15.987740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-07-30T17:03:53.476296+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-29T14:05:40.794 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-07-29T14:05:40.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 979, 'reported_epoch': 1003, 'state': 'activating', 'last_fresh': '2023-07-29T14:05:37.433490+0000', 'last_change': '2023-07-29T14:05:37.441418+0000', 'last_active': '2023-07-29T14:05:36.969928+0000', 'last_peered': '2023-07-29T14:05:35.332022+0000', 'last_clean': '2023-07-29T14:05:12.448301+0000', 'last_became_active': '2023-07-29T14:05:29.733673+0000', 'last_became_peered': '2023-07-29T14:05:29.733673+0000', 'last_unstale': '2023-07-29T14:05:37.433490+0000', 'last_undegraded': '2023-07-29T14:05:37.433490+0000', 'last_fullsized': '2023-07-29T14:05:37.433490+0000', 'mapping_epoch': 1002, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 22, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-29T13:28:12.371049+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-29T13:28:12.371049+0000', 'last_clean_scrub_stamp': '2023-07-29T13:28:12.371049+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-07-30T19:38:04.875711+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': ['3', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-29T14:05:40.794 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-07-29T14:05:40.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 944, 'reported_epoch': 1003, 'state': 'activating', 'last_fresh': '2023-07-29T14:05:37.533539+0000', 'last_change': '2023-07-29T14:05:37.544652+0000', 'last_active': '2023-07-29T14:05:36.720848+0000', 'last_peered': '2023-07-29T14:05:35.604442+0000', 'last_clean': '2023-07-29T14:04:06.556968+0000', 'last_became_active': '2023-07-29T14:05:29.660537+0000', 'last_became_peered': '2023-07-29T14:05:29.660537+0000', 'last_unstale': '2023-07-29T14:05:37.533539+0000', 'last_undegraded': '2023-07-29T14:05:37.533539+0000', 'last_fullsized': '2023-07-29T14:05:37.533539+0000', 'mapping_epoch': 1002, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 23, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-29T13:28:12.371049+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-29T13:28:12.371049+0000', 'last_clean_scrub_stamp': '2023-07-29T13:28:12.371049+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-07-30T22:42:16.790121+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': ['4', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-29T14:05:40.795 INFO:tasks.ceph.ceph_manager.ceph:PG 7.6 is not active+clean 2023-07-29T14:05:40.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 940, 'reported_epoch': 1000, 'state': 'active+undersized', 'last_fresh': '2023-07-29T14:05:34.441767+0000', 'last_change': '2023-07-29T14:05:29.759408+0000', 'last_active': '2023-07-29T14:05:34.441767+0000', 'last_peered': '2023-07-29T14:05:34.441767+0000', 'last_clean': '2023-07-29T14:04:06.482879+0000', 'last_became_active': '2023-07-29T14:05:29.759408+0000', 'last_became_peered': '2023-07-29T14:05:29.759408+0000', 'last_unstale': '2023-07-29T14:05:34.441767+0000', 'last_undegraded': '2023-07-29T14:05:34.441767+0000', 'last_fullsized': '2023-07-29T14:05:29.709468+0000', 'mapping_epoch': 995, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 26, 'last_epoch_clean': 705, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-29T13:29:15.987740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-29T13:29:15.987740+0000', 'last_clean_scrub_stamp': '2023-07-29T13:29:15.987740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-07-30T19:11:56.037546+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5], 'acting': [5], 'avail_no_missing': ['5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-29T14:05:40.795 INFO:tasks.ceph.ceph_manager.ceph:PG 7.17 is not active+clean 2023-07-29T14:05:40.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 950, 'reported_epoch': 1002, 'state': 'peering', 'last_fresh': '2023-07-29T14:05:36.299801+0000', 'last_change': '2023-07-29T14:05:36.303408+0000', 'last_active': '2023-07-29T14:05:36.299801+0000', 'last_peered': '2023-07-29T14:05:35.502600+0000', 'last_clean': '2023-07-29T14:04:06.286440+0000', 'last_became_active': '2023-07-29T14:05:29.629683+0000', 'last_became_peered': '2023-07-29T14:05:29.629683+0000', 'last_unstale': '2023-07-29T14:05:36.299801+0000', 'last_undegraded': '2023-07-29T14:05:36.299801+0000', 'last_fullsized': '2023-07-29T14:05:36.299801+0000', 'mapping_epoch': 1002, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 26, 'last_epoch_clean': 448, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-29T13:29:15.987740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-29T13:29:15.987740+0000', 'last_clean_scrub_stamp': '2023-07-29T13:29:15.987740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-07-30T18:52:32.253454+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-29T14:05:40.801 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-07-29T14:05:40.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.907+0000 7f19e671b640 10 monclient: get_monmap_and_config 2023-07-29T14:05:40.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.907+0000 7f19e671b640 10 monclient: build_initial_monmap 2023-07-29T14:05:40.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.907+0000 7f19e671b640 10 monclient: monmap: 2023-07-29T14:05:40.908 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:40.909 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:40.909 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:40.908853+0000 2023-07-29T14:05:40.909 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:40.908853+0000 2023-07-29T14:05:40.909 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:40.909 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.909 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:40.909 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:40.910 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:40.910 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:40.910 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:40.910 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.908+0000 7f19e671b640 1 Processor -- start 2023-07-29T14:05:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.908+0000 7f19e671b640 1 -- start start 2023-07-29T14:05:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.908+0000 7f19e671b640 10 monclient: init 2023-07-29T14:05:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.908+0000 7f19e671b640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.908+0000 7f19e671b640 10 monclient: _add_conns ranks=[0,4,1,3,2] 2023-07-29T14:05:40.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.908+0000 7f19e671b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f19e006d5a0 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:40.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.908+0000 7f19e671b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19e0071350 0x7f19e00717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:40.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.908+0000 7f19e671b640 10 monclient(hunting): picked mon.noname-c con 0x7f19e0071350 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:40.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.909+0000 7f19e671b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f19e0144370 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:40.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.909+0000 7f19e671b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e006d970 con 0x7f19e006d5a0 2023-07-29T14:05:40.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.909+0000 7f19e671b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e0141780 con 0x7f19e0144370 2023-07-29T14:05:40.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.909+0000 7f19e671b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:40.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.909+0000 7f19e671b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e00b1360 con 0x7f19e0071350 2023-07-29T14:05:40.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.909+0000 7f19e671b640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:40.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.909+0000 7f19e671b640 10 monclient(hunting): authenticate will time out at 3663.042130s 2023-07-29T14:05:40.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.909+0000 7f19dffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f19e006d5a0 0x7f19e0141070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51722/0 (socket says 172.21.15.40:51722) 2023-07-29T14:05:40.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.909+0000 7f19dffff640 1 -- 172.21.15.40:0/1821277730 learned_addr learned my addr 172.21.15.40:0/1821277730 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:40.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.910+0000 7f19df7fe640 1 --2- 172.21.15.40:0/1821277730 >> v2:172.21.15.45:3300/0 conn(0x7f19e0071350 0x7f19e00717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:40.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.910+0000 7f19df7fe640 10 monclient(hunting): get_auth_request con 0x7f19e0071350 auth_method 0 2023-07-29T14:05:40.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.910+0000 7f19df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.910+0000 7f19df7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.910+0000 7f19df7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.911+0000 7f19df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.911+0000 7f19df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:40.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.911+0000 7f19df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:40.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.912+0000 7f19deffd640 1 -- 172.21.15.40:0/1821277730 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701877629 0 0) 0x7f19e006d970 con 0x7f19e006d5a0 2023-07-29T14:05:40.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.912+0000 7f19deffd640 10 monclient(hunting): _negotiate 2023-07-29T14:05:40.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.912+0000 7f19deffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:40.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.912+0000 7f19deffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:40.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.912+0000 7f19deffd640 10 monclient(hunting): my global_id is 14871 2023-07-29T14:05:40.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.912+0000 7f19deffd640 1 -- 172.21.15.40:0/1821277730 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19cc003520 con 0x7f19e006d5a0 2023-07-29T14:05:40.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.913+0000 7f19df7fe640 10 monclient(hunting): handle_auth_done global_id 20320 payload 274 2023-07-29T14:05:40.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.913+0000 7f19df7fe640 1 -- 172.21.15.40:0/1821277730 >> v1:172.21.15.45:6789/0 conn(0x7f19e0144370 legacy=0x7f19e01447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:40.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.913+0000 7f19df7fe640 1 -- 172.21.15.40:0/1821277730 >> v1:172.21.15.40:6789/0 conn(0x7f19e006d5a0 legacy=0x7f19e0141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.913+0000 7f19df7fe640 10 monclient: _finish_hunting 0 2023-07-29T14:05:40.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.913+0000 7f19df7fe640 1 monclient: found mon.noname-c 2023-07-29T14:05:40.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.913+0000 7f19df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:40.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.913+0000 7f19df7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T14:05:40.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.913+0000 7f19df7fe640 1 -- 172.21.15.40:0/1821277730 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0141a40 con 0x7f19e0071350 2023-07-29T14:05:40.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.916+0000 7f19df7fe640 1 --2- 172.21.15.40:0/1821277730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e0071350 0x7f19e00717b0 secure :-1 s=READY pgs=4777 cs=0 l=1 rev1=1 crypto rx=0x7f19d00097a0 tx=0x7f19d002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e23364f0af85f92e server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:40.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 1 -- 172.21.15.40:0/1821277730 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0031830 con 0x7f19e0071350 2023-07-29T14:05:40.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:40.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:40.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 10 monclient: dump: 2023-07-29T14:05:40.923 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:40.923 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:40.923 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.923 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.923 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:40.923 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.923 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:40.923 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:40.924 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:40.924 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 10 monclient: _finish_auth 0 2023-07-29T14:05:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 1 -- 172.21.15.40:0/1821277730 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d0031990 con 0x7f19e0071350 2023-07-29T14:05:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 1 -- 172.21.15.40:0/1821277730 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0031d20 con 0x7f19e0071350 2023-07-29T14:05:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 5 monclient: authenticate success, global_id 20320 2023-07-29T14:05:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 4 set_mon_vals no callback set 2023-07-29T14:05:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:40.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T14:05:40.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19deffd640 10 monclient: dump: 2023-07-29T14:05:40.930 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:40.930 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:40.930 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.930 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.930 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:40.930 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:40.931 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 10 monclient: shutdown 2023-07-29T14:05:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 1 -- 172.21.15.40:0/1821277730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e0071350 msgr2=0x7f19e00717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 1 --2- 172.21.15.40:0/1821277730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e0071350 0x7f19e00717b0 secure :-1 s=READY pgs=4777 cs=0 l=1 rev1=1 crypto rx=0x7f19d00097a0 tx=0x7f19d002b8d0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 1 -- 172.21.15.40:0/1821277730 shutdown_connections 2023-07-29T14:05:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 1 --2- 172.21.15.40:0/1821277730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e0071350 0x7f19e00717b0 unknown :-1 s=CLOSED pgs=4777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 1 -- 172.21.15.40:0/1821277730 >> 172.21.15.40:0/1821277730 conn(0x7f19e00b4d80 msgr2=0x7f19e00b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:40.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 1 -- 172.21.15.40:0/1821277730 shutdown_connections 2023-07-29T14:05:40.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.921+0000 7f19e671b640 1 -- 172.21.15.40:0/1821277730 wait complete. 2023-07-29T14:05:40.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.922+0000 7f19e671b640 10 monclient: build_initial_monmap 2023-07-29T14:05:40.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.922+0000 7f19e671b640 10 monclient: monmap: 2023-07-29T14:05:40.941 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:40.941 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:40.941 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:40.941 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:40.941 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:40.941 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.942 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:40.942 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:40.942 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:40.942 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.922+0000 7f19e671b640 1 librados: starting msgr at 2023-07-29T14:05:40.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.922+0000 7f19e671b640 1 librados: starting objecter 2023-07-29T14:05:40.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.922+0000 7f19e671b640 1 Processor -- start 2023-07-29T14:05:40.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 1 -- start start 2023-07-29T14:05:40.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 1 librados: setting wanted keys 2023-07-29T14:05:40.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 1 librados: calling monclient init 2023-07-29T14:05:40.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 10 monclient: init 2023-07-29T14:05:40.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:40.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T14:05:40.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f19e006d5a0 0x7f19e00afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:40.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 10 monclient(hunting): picked mon.noname-c con 0x7f19e006d5a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:40.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0071350 0x7f19e0137e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:40.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 10 monclient(hunting): picked mon.noname-a con 0x7f19e0071350 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:40.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e0144370 0x7f19e01383b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:40.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 10 monclient(hunting): picked mon.noname-b con 0x7f19e0144370 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:40.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:40.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e01388f0 con 0x7f19e0071350 2023-07-29T14:05:40.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:40.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e013b390 con 0x7f19e0144370 2023-07-29T14:05:40.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:40.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e013a660 con 0x7f19e006d5a0 2023-07-29T14:05:40.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:40.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.923+0000 7f19e671b640 10 monclient(hunting): authenticate will time out at 3663.056626s 2023-07-29T14:05:40.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0071350 0x7f19e0137e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:40.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19dffff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f19e006d5a0 0x7f19e00afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:40.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19e4cb8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e0144370 0x7f19e01383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:40.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19dffff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f19e006d5a0 0x7f19e00afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49406/0 (socket says 172.21.15.40:49406) 2023-07-29T14:05:40.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19dffff640 1 -- 172.21.15.40:0/1394165507 learned_addr learned my addr 172.21.15.40:0/1394165507 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:40.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19dffff640 10 monclient(hunting): get_auth_request con 0x7f19e006d5a0 auth_method 0 2023-07-29T14:05:40.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:40.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19dffff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:40.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19dffff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:40.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19df7fe640 10 monclient(hunting): get_auth_request con 0x7f19e0071350 auth_method 0 2023-07-29T14:05:40.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:40.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19df7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:40.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19df7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:40.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19e4cb8640 10 monclient(hunting): get_auth_request con 0x7f19e0144370 auth_method 0 2023-07-29T14:05:40.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19e4cb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:40.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19e4cb8640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:40.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.924+0000 7f19e4cb8640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:40.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.925+0000 7f19dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:40.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.925+0000 7f19dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:40.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.925+0000 7f19dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:40.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.925+0000 7f19df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:40.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.925+0000 7f19df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:40.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.925+0000 7f19df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:40.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.925+0000 7f19e4cb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:40.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.925+0000 7f19e4cb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:40.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.925+0000 7f19e4cb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:40.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.927+0000 7f19dffff640 10 monclient(hunting): handle_auth_done global_id 16346 payload 824 2023-07-29T14:05:40.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.927+0000 7f19dffff640 1 -- 172.21.15.40:0/1394165507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e0144370 msgr2=0x7f19e01383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.927+0000 7f19dffff640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e0144370 0x7f19e01383b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.927+0000 7f19dffff640 1 -- 172.21.15.40:0/1394165507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0071350 msgr2=0x7f19e0137e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:40.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.927+0000 7f19dffff640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0071350 0x7f19e0137e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:40.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.927+0000 7f19dffff640 10 monclient: _finish_hunting 0 2023-07-29T14:05:40.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.927+0000 7f19dffff640 1 monclient: found mon.noname-c 2023-07-29T14:05:40.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.927+0000 7f19dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.927+0000 7f19dffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.927+0000 7f19dffff640 1 -- 172.21.15.40:0/1394165507 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d0009400 con 0x7f19e006d5a0 2023-07-29T14:05:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.930+0000 7f19dffff640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f19e006d5a0 0x7f19e00afc10 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 crypto rx=0x7f19c800c460 tx=0x7f19c800c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.930+0000 7f19dcff9640 10 client.?.objecter ms_handle_connect 0x7f19e006d5a0 2023-07-29T14:05:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.930+0000 7f19dcff9640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 1 -- 172.21.15.40:0/1394165507 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8005ce0 con 0x7f19e006d5a0 2023-07-29T14:05:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:40.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:40.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 10 monclient: dump: 2023-07-29T14:05:40.954 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:40.954 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:40.954 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.954 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.954 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:40.954 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.955 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:40.955 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:40.955 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:40.955 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 10 monclient: _finish_auth 0 2023-07-29T14:05:40.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 10 monclient: _check_auth_tickets 2023-07-29T14:05:40.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:40.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 1 -- 172.21.15.40:0/1394165507 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c8005e40 con 0x7f19e006d5a0 2023-07-29T14:05:40.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 1 -- 172.21.15.40:0/1394165507 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c80072f0 con 0x7f19e006d5a0 2023-07-29T14:05:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:40.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:40.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19dcff9640 10 monclient: dump: 2023-07-29T14:05:40.958 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:40.958 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:40.958 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.958 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:40.958 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:40.959 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:40.959 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:40.959 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:40.959 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:40.959 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:40.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19e671b640 5 monclient: authenticate success, global_id 16346 2023-07-29T14:05:40.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19e671b640 10 monclient: _renew_subs 2023-07-29T14:05:40.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19e671b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:40.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19e671b640 1 -- 172.21.15.40:0/1394165507 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e013a2f0 con 0x7f19e006d5a0 2023-07-29T14:05:40.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19de7fc640 4 set_mon_vals no callback set 2023-07-29T14:05:40.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19e671b640 10 client.16346.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:40.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19e671b640 10 monclient: _renew_subs 2023-07-29T14:05:40.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19e671b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:40.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.933+0000 7f19e671b640 1 -- 172.21.15.40:0/1394165507 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e013ad50 con 0x7f19e006d5a0 2023-07-29T14:05:40.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.934+0000 7f19e671b640 1 librados: init done 2023-07-29T14:05:40.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.935+0000 7f19e671b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:40.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.935+0000 7f19e671b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:40.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.935+0000 7f19e671b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:40.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.935+0000 7f19e671b640 1 -- 172.21.15.40:0/1394165507 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e004e000 con 0x7f19e006d5a0 2023-07-29T14:05:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.953+0000 7f19dcff9640 1 -- 172.21.15.40:0/1394165507 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f19c8007450 con 0x7f19e006d5a0 2023-07-29T14:05:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.953+0000 7f19dcff9640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f19c006bfb0 0x7f19c006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.954+0000 7f19df7fe640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f19c006bfb0 0x7f19c006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.954+0000 7f19df7fe640 10 monclient: get_auth_request con 0x7f19c006bfb0 auth_method 0 2023-07-29T14:05:40.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.954+0000 7f19df7fe640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f19c006bfb0 0x7f19c006e460 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f19d0036000 tx=0x7f19d003c040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:40.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.954+0000 7f19dcff9640 10 client.16346.objecter ms_handle_connect 0x7f19c006bfb0 2023-07-29T14:05:40.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.955+0000 7f19dcff9640 1 -- 172.21.15.40:0/1394165507 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f19c805b940 con 0x7f19e006d5a0 2023-07-29T14:05:40.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.955+0000 7f19dcff9640 10 client.16346.objecter ms_dispatch 0x7f19e0000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:40.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.955+0000 7f19dcff9640 3 client.16346.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:40.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.955+0000 7f19dcff9640 3 client.16346.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:40.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:40.955+0000 7f19dcff9640 20 client.16346.objecter dump_active .. 0 homeless 2023-07-29T14:05:41.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.210+0000 7f19dcff9640 1 -- 172.21.15.40:0/1394165507 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f19c8053d70 con 0x7f19e006d5a0 2023-07-29T14:05:41.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.210+0000 7f19dcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:41.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.210+0000 7f19dcff9640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:41.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.461+0000 7f19e671b640 10 start_mon_command cmd=[{"prefix": "status", "format": "json"}] 2023-07-29T14:05:41.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.461+0000 7f19e671b640 10 monclient: _send_command 2 [{"prefix": "status", "format": "json"}] 2023-07-29T14:05:41.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.461+0000 7f19e671b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:41.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.461+0000 7f19e671b640 1 -- 172.21.15.40:0/1394165507 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f19e00acf00 con 0x7f19e006d5a0 2023-07-29T14:05:41.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.467+0000 7f19dcff9640 1 -- 172.21.15.40:0/1394165507 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1154 (secure 0 0 0) 0x7f19c80174a0 con 0x7f19e006d5a0 2023-07-29T14:05:41.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.467+0000 7f19dcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "status", "format": "json"}] 2023-07-29T14:05:41.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.467+0000 7f19dcff9640 10 monclient: _finish_command 2 = system:0 2023-07-29T14:05:41.470 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T14:05:41.470 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","health":{"status":"HEALTH_WARN","checks":{"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":2304,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":1004,"num_osds":8,"num_up_osds":8,"osd_up_since":1690639536,"num_in_osds":8,"osd_in_since":1690637242,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":36},{"state_name":"activating","count":3},{"state_name":"active+undersized","count":1},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":3,"data_bytes":459285,"bytes_used":538042368,"bytes_avail":772556070912,"bytes_total":773094113280,"inactive_pgs_ratio":0.097560971975326538},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2023-07-29T14:05:03.619855+0000","services":{}},"progress_events":{}} 2023-07-29T14:05:41.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 10 librados: watch_flush enter 2023-07-29T14:05:41.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 10 librados: watch_flush exit 2023-07-29T14:05:41.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 20 client.16346.objecter shutdown clearing up homeless session... 2023-07-29T14:05:41.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 10 client.16346.objecter successfully canceled tick 2023-07-29T14:05:41.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 -- 172.21.15.40:0/1394165507 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f19c006bfb0 msgr2=0x7f19c006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:41.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f19c006bfb0 0x7f19c006e460 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f19d0036000 tx=0x7f19d003c040 comp rx=0 tx=0).stop 2023-07-29T14:05:41.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 10 monclient: shutdown 2023-07-29T14:05:41.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:41.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 -- 172.21.15.40:0/1394165507 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f19e006d5a0 msgr2=0x7f19e00afc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:41.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f19e006d5a0 0x7f19e00afc10 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 crypto rx=0x7f19c800c460 tx=0x7f19c800c920 comp rx=0 tx=0).stop 2023-07-29T14:05:41.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 -- 172.21.15.40:0/1394165507 shutdown_connections 2023-07-29T14:05:41.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f19c006bfb0 0x7f19c006e460 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e0144370 0x7f19e01383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0071350 0x7f19e0137e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 --2- 172.21.15.40:0/1394165507 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f19e006d5a0 0x7f19e00afc10 unknown :-1 s=CLOSED pgs=3218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 -- 172.21.15.40:0/1394165507 >> 172.21.15.40:0/1394165507 conn(0x7f19e00b4d80 msgr2=0x7f19e00b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 -- 172.21.15.40:0/1394165507 shutdown_connections 2023-07-29T14:05:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 -- 172.21.15.40:0/1394165507 wait complete. 2023-07-29T14:05:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.469+0000 7f19e671b640 1 librados: shutdown 2023-07-29T14:05:41.481 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-07-29T14:05:41.482 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-29T14:05:41.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.589+0000 7f2ed1697640 10 monclient: get_monmap_and_config 2023-07-29T14:05:41.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.589+0000 7f2ed1697640 10 monclient: build_initial_monmap 2023-07-29T14:05:41.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.589+0000 7f2ed1697640 10 monclient: monmap: 2023-07-29T14:05:41.590 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:41.591 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:41.591 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:41.590623+0000 2023-07-29T14:05:41.591 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:41.590623+0000 2023-07-29T14:05:41.591 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:41.591 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:41.591 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:41.591 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:41.591 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:41.592 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:41.592 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:41.592 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 1 Processor -- start 2023-07-29T14:05:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 1 -- start start 2023-07-29T14:05:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 10 monclient: init 2023-07-29T14:05:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 10 monclient: _add_conns ranks=[4,3,0,1,2] 2023-07-29T14:05:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ecc0a86c0 0x7f2ecc140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 10 monclient(hunting): picked mon.noname-c con 0x7f2ecc0a86c0 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2ecc1410e0 0x7f2ecc13dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 10 monclient(hunting): picked mon.noname-a con 0x7f2ecc1410e0 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2ecc0b2f60 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ecc0a8a90 con 0x7f2ecc0b2f60 2023-07-29T14:05:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc0b3330 con 0x7f2ecc1410e0 2023-07-29T14:05:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc0a98e0 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:41.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.590+0000 7f2ed1697640 10 monclient(hunting): authenticate will time out at 3663.723756s 2023-07-29T14:05:41.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2eca7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2ecc1410e0 0x7f2ecc13dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:41.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2ecaffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ecc0a86c0 0x7f2ecc140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:41.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2eca7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2ecc1410e0 0x7f2ecc13dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53768/0 (socket says 172.21.15.40:53768) 2023-07-29T14:05:41.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2eca7fc640 1 -- 172.21.15.40:0/45207868 learned_addr learned my addr 172.21.15.40:0/45207868 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2eca7fc640 10 monclient(hunting): get_auth_request con 0x7f2ecc1410e0 auth_method 0 2023-07-29T14:05:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2eca7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2eca7fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2eca7fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2ecaffd640 10 monclient(hunting): get_auth_request con 0x7f2ecc0a86c0 auth_method 0 2023-07-29T14:05:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2ecaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2ecaffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.591+0000 7f2ecaffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.592+0000 7f2ecaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.592+0000 7f2ecaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.592+0000 7f2ecaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.592+0000 7f2eca7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.592+0000 7f2eca7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.592+0000 7f2eca7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.594+0000 7f2ecaffd640 10 monclient(hunting): handle_auth_done global_id 20326 payload 274 2023-07-29T14:05:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.594+0000 7f2ecaffd640 1 -- 172.21.15.40:0/45207868 >> v2:172.21.15.40:3300/0 conn(0x7f2ecc1410e0 msgr2=0x7f2ecc13dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.594+0000 7f2ecaffd640 1 --2- 172.21.15.40:0/45207868 >> v2:172.21.15.40:3300/0 conn(0x7f2ecc1410e0 0x7f2ecc13dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.594+0000 7f2ecaffd640 1 -- 172.21.15.40:0/45207868 >> v1:172.21.15.40:6789/0 conn(0x7f2ecc0b2f60 legacy=0x7f2ecc144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.594+0000 7f2ecaffd640 10 monclient: _finish_hunting 0 2023-07-29T14:05:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.594+0000 7f2ecaffd640 1 monclient: found mon.noname-c 2023-07-29T14:05:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.594+0000 7f2ecaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.594+0000 7f2ecaffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T14:05:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.594+0000 7f2ecaffd640 1 -- 172.21.15.40:0/45207868 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc0a9bc0 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.596+0000 7f2ecaffd640 1 --2- 172.21.15.40:0/45207868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ecc0a86c0 0x7f2ecc140ba0 secure :-1 s=READY pgs=4778 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0b7e40 tx=0x7f2ec002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25b8e31483ce0e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 1 -- 172.21.15.40:0/45207868 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec002e070 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 10 monclient: dump: 2023-07-29T14:05:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:41.607 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 10 monclient: _finish_auth 0 2023-07-29T14:05:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 10 monclient: _check_auth_tickets 2023-07-29T14:05:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 1 -- 172.21.15.40:0/45207868 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec00315a0 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 1 -- 172.21.15.40:0/45207868 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0031930 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T14:05:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ec9ffb640 10 monclient: dump: 2023-07-29T14:05:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:41.609 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:41.609 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:41.609 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:41.610 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ed1697640 5 monclient: authenticate success, global_id 20326 2023-07-29T14:05:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ed1697640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.600+0000 7f2ed1697640 4 set_mon_vals no callback set 2023-07-29T14:05:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.601+0000 7f2ed1697640 10 monclient: shutdown 2023-07-29T14:05:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.601+0000 7f2ed1697640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.601+0000 7f2ed1697640 1 -- 172.21.15.40:0/45207868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ecc0a86c0 msgr2=0x7f2ecc140ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.601+0000 7f2ed1697640 1 --2- 172.21.15.40:0/45207868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ecc0a86c0 0x7f2ecc140ba0 secure :-1 s=READY pgs=4778 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0b7e40 tx=0x7f2ec002b840 comp rx=0 tx=0).stop 2023-07-29T14:05:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.601+0000 7f2ed1697640 1 -- 172.21.15.40:0/45207868 shutdown_connections 2023-07-29T14:05:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.601+0000 7f2ed1697640 1 --2- 172.21.15.40:0/45207868 >> v2:172.21.15.40:3300/0 conn(0x7f2ecc1410e0 0x7f2ecc13dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.601+0000 7f2ed1697640 1 --2- 172.21.15.40:0/45207868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ecc0a86c0 0x7f2ecc140ba0 unknown :-1 s=CLOSED pgs=4778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.601+0000 7f2ed1697640 1 -- 172.21.15.40:0/45207868 >> 172.21.15.40:0/45207868 conn(0x7f2ecc065920 msgr2=0x7f2ecc065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.601+0000 7f2ed1697640 1 -- 172.21.15.40:0/45207868 shutdown_connections 2023-07-29T14:05:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.601+0000 7f2ed1697640 1 -- 172.21.15.40:0/45207868 wait complete. 2023-07-29T14:05:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.602+0000 7f2ed1697640 10 monclient: build_initial_monmap 2023-07-29T14:05:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.602+0000 7f2ed1697640 10 monclient: monmap: 2023-07-29T14:05:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:41.613 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:41.613 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:41.613 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:41.613 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:41.613 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:41.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.602+0000 7f2ed1697640 1 librados: starting msgr at 2023-07-29T14:05:41.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.602+0000 7f2ed1697640 1 librados: starting objecter 2023-07-29T14:05:41.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.602+0000 7f2ed1697640 1 Processor -- start 2023-07-29T14:05:41.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.602+0000 7f2ed1697640 1 -- start start 2023-07-29T14:05:41.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.602+0000 7f2ed1697640 1 librados: setting wanted keys 2023-07-29T14:05:41.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.602+0000 7f2ed1697640 1 librados: calling monclient init 2023-07-29T14:05:41.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.602+0000 7f2ed1697640 10 monclient: init 2023-07-29T14:05:41.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:41.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T14:05:41.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0a86c0 0x7f2ecc0ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:41.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 10 monclient(hunting): picked mon.noname-a con 0x7f2ecc0a86c0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:41.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ecc0b2f60 0x7f2ecc0ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:41.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 10 monclient(hunting): picked mon.noname-b con 0x7f2ecc0b2f60 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:41.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2ecc1410e0 0x7f2ecc13a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:41.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 10 monclient(hunting): picked mon.noname-c con 0x7f2ecc1410e0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:41.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:41.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0af2b0 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:41.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0abb80 con 0x7f2ecc0b2f60 2023-07-29T14:05:41.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:41.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0ad980 con 0x7f2ecc1410e0 2023-07-29T14:05:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ed1697640 10 monclient(hunting): authenticate will time out at 3663.736375s 2023-07-29T14:05:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ecaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0a86c0 0x7f2ecc0ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.603+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2ecc1410e0 0x7f2ecc13a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2eca7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ecc0b2f60 0x7f2ecc0ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0a86c0 0x7f2ecc0ac9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53784/0 (socket says 172.21.15.40:53784) 2023-07-29T14:05:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecaffd640 1 -- 172.21.15.40:0/4006007555 learned_addr learned my addr 172.21.15.40:0/4006007555 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecaffd640 10 monclient(hunting): get_auth_request con 0x7f2ecc0a86c0 auth_method 0 2023-07-29T14:05:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecaffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecaffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecb7fe640 10 monclient(hunting): get_auth_request con 0x7f2ecc1410e0 auth_method 0 2023-07-29T14:05:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecb7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecb7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2eca7fc640 10 monclient(hunting): get_auth_request con 0x7f2ecc0b2f60 auth_method 0 2023-07-29T14:05:41.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2eca7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:41.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2eca7fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:41.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2eca7fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:41.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:41.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:41.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:41.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:41.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:41.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2ecb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:41.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2eca7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:41.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2eca7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:41.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.604+0000 7f2eca7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:41.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.607+0000 7f2ecaffd640 10 monclient(hunting): handle_auth_done global_id 14880 payload 824 2023-07-29T14:05:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.607+0000 7f2ecaffd640 1 -- 172.21.15.40:0/4006007555 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2ecc1410e0 msgr2=0x7f2ecc13a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.607+0000 7f2ecaffd640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2ecc1410e0 0x7f2ecc13a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.607+0000 7f2ecaffd640 1 -- 172.21.15.40:0/4006007555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ecc0b2f60 msgr2=0x7f2ecc0ad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.607+0000 7f2ecaffd640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ecc0b2f60 0x7f2ecc0ad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.607+0000 7f2ecaffd640 10 monclient: _finish_hunting 0 2023-07-29T14:05:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.607+0000 7f2ecaffd640 1 monclient: found mon.noname-a 2023-07-29T14:05:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.607+0000 7f2ecaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.607+0000 7f2ecaffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T14:05:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.607+0000 7f2ecaffd640 1 -- 172.21.15.40:0/4006007555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec0009400 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.607+0000 7f2eca7fc640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ecc0b2f60 0x7f2ecc0ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-29T14:05:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.610+0000 7f2ecaffd640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0a86c0 0x7f2ecc0ac9c0 secure :-1 s=READY pgs=3203 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009890 tx=0x7f2ec00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.610+0000 7f2eb37fe640 10 client.?.objecter ms_handle_connect 0x7f2ecc0a86c0 2023-07-29T14:05:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.610+0000 7f2eb37fe640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 1 -- 172.21.15.40:0/4006007555 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0032840 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T14:05:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 10 monclient: dump: 2023-07-29T14:05:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:41.628 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:41.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 10 monclient: _finish_auth 0 2023-07-29T14:05:41.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 10 monclient: _check_auth_tickets 2023-07-29T14:05:41.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:41.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 1 -- 172.21.15.40:0/4006007555 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec0031f00 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2ed1697640 5 monclient: authenticate success, global_id 14880 2023-07-29T14:05:41.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2ed1697640 10 monclient: _renew_subs 2023-07-29T14:05:41.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2ed1697640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:41.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2ed1697640 1 -- 172.21.15.40:0/4006007555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ecc0adba0 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:41.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 1 -- 172.21.15.40:0/4006007555 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0032be0 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:41.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:41.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2eb37fe640 10 monclient: dump: 2023-07-29T14:05:41.632 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:41.632 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:41.632 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:41.632 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:41.632 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:41.632 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:41.633 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:41.633 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:41.633 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:41.633 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:41.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2ec8ff9640 4 set_mon_vals no callback set 2023-07-29T14:05:41.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2ed1697640 10 client.14880.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:41.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2ed1697640 10 monclient: _renew_subs 2023-07-29T14:05:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2ed1697640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2ed1697640 1 -- 172.21.15.40:0/4006007555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc13a770 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.614+0000 7f2ed1697640 1 librados: init done 2023-07-29T14:05:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.615+0000 7f2ed1697640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.615+0000 7f2ed1697640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.615+0000 7f2ed1697640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.615+0000 7f2ed1697640 1 -- 172.21.15.40:0/4006007555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ecc04e000 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.870+0000 7f2eb37fe640 1 -- 172.21.15.40:0/4006007555 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f2ec0032d40 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.870+0000 7f2eb37fe640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2eb806bfb0 0x7f2eb806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:41.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.871+0000 7f2eca7fc640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2eb806bfb0 0x7f2eb806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.871+0000 7f2eca7fc640 10 monclient: get_auth_request con 0x7f2eb806bfb0 auth_method 0 2023-07-29T14:05:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.872+0000 7f2eb37fe640 10 client.14880.objecter ms_handle_connect 0x7f2eb806bfb0 2023-07-29T14:05:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.872+0000 7f2eca7fc640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2eb806bfb0 0x7f2eb806e460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0aea00 tx=0x7f2ebc009240 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:41.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.883+0000 7f2eb37fe640 1 -- 172.21.15.40:0/4006007555 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f2ec007d970 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.883+0000 7f2eb37fe640 10 client.14880.objecter ms_dispatch 0x7f2ecc000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:41.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.883+0000 7f2eb37fe640 3 client.14880.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:41.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.883+0000 7f2eb37fe640 3 client.14880.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:41.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.883+0000 7f2eb37fe640 20 client.14880.objecter dump_active .. 0 homeless 2023-07-29T14:05:41.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.883+0000 7f2eb37fe640 1 -- 172.21.15.40:0/4006007555 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f2ec0075da0 con 0x7f2ecc0a86c0 2023-07-29T14:05:41.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.883+0000 7f2eb37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:41.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:41.883+0000 7f2eb37fe640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:41.995 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:37.327 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:41.995 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:37.329 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:42.006 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:37.341 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:42.040 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:37.375 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:42.043 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:37.378 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:42.047 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:37.331 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:42.048 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:37.332 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:42.056 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:37.392 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:42.059 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:37.344 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:42.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.102+0000 7f2ed1697640 1 -- 172.21.15.40:0/4006007555 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ecc0b2a40 con 0x7f2eb806bfb0 2023-07-29T14:05:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.107+0000 7f2eb37fe640 1 -- 172.21.15.40:0/4006007555 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152034 (secure 0 0 0) 0x7f2ebc011e60 con 0x7f2eb806bfb0 2023-07-29T14:05:42.109 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T14:05:42.110 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-29T14:05:42.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 10 librados: watch_flush enter 2023-07-29T14:05:42.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 10 librados: watch_flush exit 2023-07-29T14:05:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 20 client.14880.objecter shutdown clearing up homeless session... 2023-07-29T14:05:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 10 client.14880.objecter successfully canceled tick 2023-07-29T14:05:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 -- 172.21.15.40:0/4006007555 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2eb806bfb0 msgr2=0x7f2eb806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2eb806bfb0 0x7f2eb806e460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0aea00 tx=0x7f2ebc009240 comp rx=0 tx=0).stop 2023-07-29T14:05:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 10 monclient: shutdown 2023-07-29T14:05:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 -- 172.21.15.40:0/4006007555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0a86c0 msgr2=0x7f2ecc0ac9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:42.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0a86c0 0x7f2ecc0ac9c0 secure :-1 s=READY pgs=3203 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009890 tx=0x7f2ec00046c0 comp rx=0 tx=0).stop 2023-07-29T14:05:42.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 -- 172.21.15.40:0/4006007555 shutdown_connections 2023-07-29T14:05:42.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f2eb806bfb0 0x7f2eb806e460 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:42.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ecc0b2f60 0x7f2ecc0ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:42.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2ecc1410e0 0x7f2ecc13a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:42.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 --2- 172.21.15.40:0/4006007555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0a86c0 0x7f2ecc0ac9c0 unknown :-1 s=CLOSED pgs=3203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:42.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 -- 172.21.15.40:0/4006007555 >> 172.21.15.40:0/4006007555 conn(0x7f2ecc065920 msgr2=0x7f2ecc13de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 -- 172.21.15.40:0/4006007555 shutdown_connections 2023-07-29T14:05:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 -- 172.21.15.40:0/4006007555 wait complete. 2023-07-29T14:05:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:42.109+0000 7f2ed1697640 1 librados: shutdown 2023-07-29T14:05:42.122 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":2132,"stamp":"2023-07-29T14:05:41.629459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":525368,"kb_used_data":53624,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754449352,"statfs":{"total":773094113280,"available":772556136448,"internally_reserved":0,"allocated":54910976,"data_stored":30218056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":374,"apply_latency_ms":374,"commit_latency_ns":374000000,"apply_latency_ns":374000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4000299,"num_objects":-44,"num_object_clones":-1,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44,"num_whiteouts":0,"num_read":-31,"num_read_kb":-11,"num_write":-71,"num_write_kb":-3969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001162"},"pg_stats":[{"pgid":"7.1b","version":"0'0","reported_seq":968,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.762797+0000","last_change":"2023-07-29T13:56:55.662556+0000","last_active":"2023-07-29T14:05:36.762797+0000","last_peered":"2023-07-29T14:05:36.762797+0000","last_clean":"2023-07-29T14:05:36.762797+0000","last_became_active":"2023-07-29T13:56:55.656930+0000","last_became_peered":"2023-07-29T13:56:55.656930+0000","last_unstale":"2023-07-29T14:05:36.762797+0000","last_undegraded":"2023-07-29T14:05:36.762797+0000","last_fullsized":"2023-07-29T14:05:36.762797+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:52:51.321547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":977,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.016088+0000","last_change":"2023-07-29T13:41:38.328763+0000","last_active":"2023-07-29T14:05:37.016088+0000","last_peered":"2023-07-29T14:05:37.016088+0000","last_clean":"2023-07-29T14:05:37.016088+0000","last_became_active":"2023-07-29T13:41:38.322679+0000","last_became_peered":"2023-07-29T13:41:38.322679+0000","last_unstale":"2023-07-29T14:05:37.016088+0000","last_undegraded":"2023-07-29T14:05:37.016088+0000","last_fullsized":"2023-07-29T14:05:37.016088+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:40:55.886733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":964,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.997533+0000","last_change":"2023-07-29T13:29:52.105383+0000","last_active":"2023-07-29T14:05:36.997533+0000","last_peered":"2023-07-29T14:05:36.997533+0000","last_clean":"2023-07-29T14:05:36.997533+0000","last_became_active":"2023-07-29T13:29:52.099163+0000","last_became_peered":"2023-07-29T13:29:52.099163+0000","last_unstale":"2023-07-29T14:05:36.997533+0000","last_undegraded":"2023-07-29T14:05:36.997533+0000","last_fullsized":"2023-07-29T14:05:36.997533+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:03:36.535099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":930,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.709810+0000","last_change":"2023-07-29T13:29:41.816664+0000","last_active":"2023-07-29T14:05:36.709810+0000","last_peered":"2023-07-29T14:05:36.709810+0000","last_clean":"2023-07-29T14:05:36.709810+0000","last_became_active":"2023-07-29T13:29:41.782506+0000","last_became_peered":"2023-07-29T13:29:41.782506+0000","last_unstale":"2023-07-29T14:05:36.709810+0000","last_undegraded":"2023-07-29T14:05:36.709810+0000","last_fullsized":"2023-07-29T14:05:36.709810+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:20:57.566653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":947,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.066403+0000","last_change":"2023-07-29T14:05:38.066403+0000","last_active":"2023-07-29T14:05:38.066403+0000","last_peered":"2023-07-29T14:05:38.066403+0000","last_clean":"2023-07-29T14:05:38.066403+0000","last_became_active":"2023-07-29T14:05:38.060277+0000","last_became_peered":"2023-07-29T14:05:38.060277+0000","last_unstale":"2023-07-29T14:05:38.066403+0000","last_undegraded":"2023-07-29T14:05:38.066403+0000","last_fullsized":"2023-07-29T14:05:38.066403+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:50:25.119138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":913,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.286005+0000","last_change":"2023-07-29T13:53:19.606247+0000","last_active":"2023-07-29T14:05:36.286005+0000","last_peered":"2023-07-29T14:05:36.286005+0000","last_clean":"2023-07-29T14:05:36.286005+0000","last_became_active":"2023-07-29T13:53:19.600386+0000","last_became_peered":"2023-07-29T13:53:19.600386+0000","last_unstale":"2023-07-29T14:05:36.286005+0000","last_undegraded":"2023-07-29T14:05:36.286005+0000","last_fullsized":"2023-07-29T14:05:36.286005+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:46:03.055229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":937,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.571721+0000","last_change":"2023-07-29T13:56:55.406251+0000","last_active":"2023-07-29T14:05:37.571721+0000","last_peered":"2023-07-29T14:05:37.571721+0000","last_clean":"2023-07-29T14:05:37.571721+0000","last_became_active":"2023-07-29T13:56:55.400591+0000","last_became_peered":"2023-07-29T13:56:55.400591+0000","last_unstale":"2023-07-29T14:05:37.571721+0000","last_undegraded":"2023-07-29T14:05:37.571721+0000","last_fullsized":"2023-07-29T14:05:37.571721+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:40:48.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":954,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.369590+0000","last_change":"2023-07-29T13:29:31.089676+0000","last_active":"2023-07-29T14:05:36.369590+0000","last_peered":"2023-07-29T14:05:36.369590+0000","last_clean":"2023-07-29T14:05:36.369590+0000","last_became_active":"2023-07-29T13:29:31.072042+0000","last_became_peered":"2023-07-29T13:29:31.072042+0000","last_unstale":"2023-07-29T14:05:36.369590+0000","last_undegraded":"2023-07-29T14:05:36.369590+0000","last_fullsized":"2023-07-29T14:05:36.369590+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:28:28.141003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":912,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.243324+0000","last_change":"2023-07-29T13:53:19.677067+0000","last_active":"2023-07-29T14:05:36.243324+0000","last_peered":"2023-07-29T14:05:36.243324+0000","last_clean":"2023-07-29T14:05:36.243324+0000","last_became_active":"2023-07-29T13:53:19.671215+0000","last_became_peered":"2023-07-29T13:53:19.671215+0000","last_unstale":"2023-07-29T14:05:36.243324+0000","last_undegraded":"2023-07-29T14:05:36.243324+0000","last_fullsized":"2023-07-29T14:05:36.243324+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:51:18.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":985,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.445062+0000","last_change":"2023-07-29T13:56:55.517147+0000","last_active":"2023-07-29T14:05:35.445062+0000","last_peered":"2023-07-29T14:05:35.445062+0000","last_clean":"2023-07-29T14:05:35.445062+0000","last_became_active":"2023-07-29T13:56:55.511490+0000","last_became_peered":"2023-07-29T13:56:55.511490+0000","last_unstale":"2023-07-29T14:05:35.445062+0000","last_undegraded":"2023-07-29T14:05:35.445062+0000","last_fullsized":"2023-07-29T14:05:35.445062+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:26.433868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":962,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.507347+0000","last_change":"2023-07-29T14:05:37.515837+0000","last_active":"2023-07-29T14:05:36.327038+0000","last_peered":"2023-07-29T14:05:35.596454+0000","last_clean":"2023-07-29T14:04:22.472067+0000","last_became_active":"2023-07-29T14:05:29.733569+0000","last_became_peered":"2023-07-29T14:05:29.733569+0000","last_unstale":"2023-07-29T14:05:37.507347+0000","last_undegraded":"2023-07-29T14:05:37.507347+0000","last_fullsized":"2023-07-29T14:05:37.507347+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:03:53.476296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":913,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.470943+0000","last_change":"2023-07-29T13:53:19.644608+0000","last_active":"2023-07-29T14:05:37.470943+0000","last_peered":"2023-07-29T14:05:37.470943+0000","last_clean":"2023-07-29T14:05:37.470943+0000","last_became_active":"2023-07-29T13:53:19.637473+0000","last_became_peered":"2023-07-29T13:53:19.637473+0000","last_unstale":"2023-07-29T14:05:37.470943+0000","last_undegraded":"2023-07-29T14:05:37.470943+0000","last_fullsized":"2023-07-29T14:05:37.470943+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:53:02.384899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":932,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.746967+0000","last_change":"2023-07-29T13:29:33.530600+0000","last_active":"2023-07-29T14:05:36.746967+0000","last_peered":"2023-07-29T14:05:36.746967+0000","last_clean":"2023-07-29T14:05:36.746967+0000","last_became_active":"2023-07-29T13:29:33.515255+0000","last_became_peered":"2023-07-29T13:29:33.515255+0000","last_unstale":"2023-07-29T14:05:36.746967+0000","last_undegraded":"2023-07-29T14:05:36.746967+0000","last_fullsized":"2023-07-29T14:05:36.746967+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:21:56.794070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":979,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.433490+0000","last_change":"2023-07-29T14:05:37.441418+0000","last_active":"2023-07-29T14:05:36.969928+0000","last_peered":"2023-07-29T14:05:35.332022+0000","last_clean":"2023-07-29T14:05:12.448301+0000","last_became_active":"2023-07-29T14:05:29.733673+0000","last_became_peered":"2023-07-29T14:05:29.733673+0000","last_unstale":"2023-07-29T14:05:37.433490+0000","last_undegraded":"2023-07-29T14:05:37.433490+0000","last_fullsized":"2023-07-29T14:05:37.433490+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:38:04.875711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":939,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.573766+0000","last_change":"2023-07-29T13:56:55.146603+0000","last_active":"2023-07-29T14:05:37.573766+0000","last_peered":"2023-07-29T14:05:37.573766+0000","last_clean":"2023-07-29T14:05:37.573766+0000","last_became_active":"2023-07-29T13:56:55.139167+0000","last_became_peered":"2023-07-29T13:56:55.139167+0000","last_unstale":"2023-07-29T14:05:37.573766+0000","last_undegraded":"2023-07-29T14:05:37.573766+0000","last_fullsized":"2023-07-29T14:05:37.573766+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:56:23.477235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":944,"reported_epoch":1003,"state":"activating","last_fresh":"2023-07-29T14:05:37.533539+0000","last_change":"2023-07-29T14:05:37.544652+0000","last_active":"2023-07-29T14:05:36.720848+0000","last_peered":"2023-07-29T14:05:35.604442+0000","last_clean":"2023-07-29T14:04:06.556968+0000","last_became_active":"2023-07-29T14:05:29.660537+0000","last_became_peered":"2023-07-29T14:05:29.660537+0000","last_unstale":"2023-07-29T14:05:37.533539+0000","last_undegraded":"2023-07-29T14:05:37.533539+0000","last_fullsized":"2023-07-29T14:05:37.533539+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:42:16.790121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":987,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.436816+0000","last_change":"2023-07-29T13:56:55.073408+0000","last_active":"2023-07-29T14:05:35.436816+0000","last_peered":"2023-07-29T14:05:35.436816+0000","last_clean":"2023-07-29T14:05:35.436816+0000","last_became_active":"2023-07-29T13:56:55.065724+0000","last_became_peered":"2023-07-29T13:56:55.065724+0000","last_unstale":"2023-07-29T14:05:35.436816+0000","last_undegraded":"2023-07-29T14:05:35.436816+0000","last_fullsized":"2023-07-29T14:05:35.436816+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:42:58.825948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":971,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.050103+0000","last_change":"2023-07-29T13:28:14.434312+0000","last_active":"2023-07-29T14:05:37.050103+0000","last_peered":"2023-07-29T14:05:37.050103+0000","last_clean":"2023-07-29T14:05:37.050103+0000","last_became_active":"2023-07-29T13:28:14.423044+0000","last_became_peered":"2023-07-29T13:28:14.423044+0000","last_unstale":"2023-07-29T14:05:37.050103+0000","last_undegraded":"2023-07-29T14:05:37.050103+0000","last_fullsized":"2023-07-29T14:05:37.050103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:56:27.865715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1","version":"70'7","reported_seq":983,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.310515+0000","last_change":"2023-07-29T13:41:38.528285+0000","last_active":"2023-07-29T14:05:36.310515+0000","last_peered":"2023-07-29T14:05:36.310515+0000","last_clean":"2023-07-29T14:05:36.310515+0000","last_became_active":"2023-07-29T13:41:38.521847+0000","last_became_peered":"2023-07-29T13:41:38.521847+0000","last_unstale":"2023-07-29T14:05:36.310515+0000","last_undegraded":"2023-07-29T14:05:36.310515+0000","last_fullsized":"2023-07-29T14:05:36.310515+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:08:42.836378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":945,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.566657+0000","last_change":"2023-07-29T13:56:55.263959+0000","last_active":"2023-07-29T14:05:37.566657+0000","last_peered":"2023-07-29T14:05:37.566657+0000","last_clean":"2023-07-29T14:05:37.566657+0000","last_became_active":"2023-07-29T13:56:55.258241+0000","last_became_peered":"2023-07-29T13:56:55.258241+0000","last_unstale":"2023-07-29T14:05:37.566657+0000","last_undegraded":"2023-07-29T14:05:37.566657+0000","last_fullsized":"2023-07-29T14:05:37.566657+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:42.517469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":966,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.065287+0000","last_change":"2023-07-29T13:29:52.230014+0000","last_active":"2023-07-29T14:05:37.065287+0000","last_peered":"2023-07-29T14:05:37.065287+0000","last_clean":"2023-07-29T14:05:37.065287+0000","last_became_active":"2023-07-29T13:29:52.224467+0000","last_became_peered":"2023-07-29T13:29:52.224467+0000","last_unstale":"2023-07-29T14:05:37.065287+0000","last_undegraded":"2023-07-29T14:05:37.065287+0000","last_fullsized":"2023-07-29T14:05:37.065287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:51:03.019114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":972,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.014430+0000","last_change":"2023-07-29T13:28:14.652586+0000","last_active":"2023-07-29T14:05:37.014430+0000","last_peered":"2023-07-29T14:05:37.014430+0000","last_clean":"2023-07-29T14:05:37.014430+0000","last_became_active":"2023-07-29T13:28:14.642023+0000","last_became_peered":"2023-07-29T13:28:14.642023+0000","last_unstale":"2023-07-29T14:05:37.014430+0000","last_undegraded":"2023-07-29T14:05:37.014430+0000","last_fullsized":"2023-07-29T14:05:37.014430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:25:23.334956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":933,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.764552+0000","last_change":"2023-07-29T13:29:41.869711+0000","last_active":"2023-07-29T14:05:36.764552+0000","last_peered":"2023-07-29T14:05:36.764552+0000","last_clean":"2023-07-29T14:05:36.764552+0000","last_became_active":"2023-07-29T13:29:41.844314+0000","last_became_peered":"2023-07-29T13:29:41.844314+0000","last_unstale":"2023-07-29T14:05:36.764552+0000","last_undegraded":"2023-07-29T14:05:36.764552+0000","last_fullsized":"2023-07-29T14:05:36.764552+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:41:40.503015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":963,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.209438+0000","last_change":"2023-07-29T13:56:55.159956+0000","last_active":"2023-07-29T14:05:36.209438+0000","last_peered":"2023-07-29T14:05:36.209438+0000","last_clean":"2023-07-29T14:05:36.209438+0000","last_became_active":"2023-07-29T13:56:55.150514+0000","last_became_peered":"2023-07-29T13:56:55.150514+0000","last_unstale":"2023-07-29T14:05:36.209438+0000","last_undegraded":"2023-07-29T14:05:36.209438+0000","last_fullsized":"2023-07-29T14:05:36.209438+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:16:38.329000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1005,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.048452+0000","last_change":"2023-07-29T13:56:55.615681+0000","last_active":"2023-07-29T14:05:37.048452+0000","last_peered":"2023-07-29T14:05:37.048452+0000","last_clean":"2023-07-29T14:05:37.048452+0000","last_became_active":"2023-07-29T13:56:55.609414+0000","last_became_peered":"2023-07-29T13:56:55.609414+0000","last_unstale":"2023-07-29T14:05:37.048452+0000","last_undegraded":"2023-07-29T14:05:37.048452+0000","last_fullsized":"2023-07-29T14:05:37.048452+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:36:30.601005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1012,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.999968+0000","last_change":"2023-07-29T13:56:55.550112+0000","last_active":"2023-07-29T14:05:36.999968+0000","last_peered":"2023-07-29T14:05:36.999968+0000","last_clean":"2023-07-29T14:05:36.999968+0000","last_became_active":"2023-07-29T13:56:55.543861+0000","last_became_peered":"2023-07-29T13:56:55.543861+0000","last_unstale":"2023-07-29T14:05:36.999968+0000","last_undegraded":"2023-07-29T14:05:36.999968+0000","last_fullsized":"2023-07-29T14:05:36.999968+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:57:36.179121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":951,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.620825+0000","last_change":"2023-07-29T14:05:37.613527+0000","last_active":"2023-07-29T14:05:37.620825+0000","last_peered":"2023-07-29T14:05:37.620825+0000","last_clean":"2023-07-29T14:05:37.620825+0000","last_became_active":"2023-07-29T14:05:37.607023+0000","last_became_peered":"2023-07-29T14:05:37.607023+0000","last_unstale":"2023-07-29T14:05:37.620825+0000","last_undegraded":"2023-07-29T14:05:37.620825+0000","last_fullsized":"2023-07-29T14:05:37.620825+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:07:39.130065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":972,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.010320+0000","last_change":"2023-07-29T13:28:14.713485+0000","last_active":"2023-07-29T14:05:37.010320+0000","last_peered":"2023-07-29T14:05:37.010320+0000","last_clean":"2023-07-29T14:05:37.010320+0000","last_became_active":"2023-07-29T13:28:14.708042+0000","last_became_peered":"2023-07-29T13:28:14.708042+0000","last_unstale":"2023-07-29T14:05:37.010320+0000","last_undegraded":"2023-07-29T14:05:37.010320+0000","last_fullsized":"2023-07-29T14:05:37.010320+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:25:56.525512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":1099,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.438875+0000","last_change":"2023-07-29T13:56:55.164457+0000","last_active":"2023-07-29T14:05:35.438875+0000","last_peered":"2023-07-29T14:05:35.438875+0000","last_clean":"2023-07-29T14:05:35.438875+0000","last_became_active":"2023-07-29T13:56:55.157702+0000","last_became_peered":"2023-07-29T13:56:55.157702+0000","last_unstale":"2023-07-29T14:05:35.438875+0000","last_undegraded":"2023-07-29T14:05:35.438875+0000","last_fullsized":"2023-07-29T14:05:35.438875+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_clean_scrub_stamp":"2023-07-29T13:28:06.434639+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:06:43.352094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":995,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.367084+0000","last_change":"2023-07-29T13:56:55.347022+0000","last_active":"2023-07-29T14:05:36.367084+0000","last_peered":"2023-07-29T14:05:36.367084+0000","last_clean":"2023-07-29T14:05:36.367084+0000","last_became_active":"2023-07-29T13:56:55.341426+0000","last_became_peered":"2023-07-29T13:56:55.341426+0000","last_unstale":"2023-07-29T14:05:36.367084+0000","last_undegraded":"2023-07-29T14:05:36.367084+0000","last_fullsized":"2023-07-29T14:05:36.367084+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:46:20.874037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":945,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.466789+0000","last_change":"2023-07-29T13:41:38.063359+0000","last_active":"2023-07-29T14:05:35.466789+0000","last_peered":"2023-07-29T14:05:35.466789+0000","last_clean":"2023-07-29T14:05:35.466789+0000","last_became_active":"2023-07-29T13:41:38.057649+0000","last_became_peered":"2023-07-29T13:41:38.057649+0000","last_unstale":"2023-07-29T14:05:35.466789+0000","last_undegraded":"2023-07-29T14:05:35.466789+0000","last_fullsized":"2023-07-29T14:05:35.466789+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:36:46.868780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":925,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.290603+0000","last_change":"2023-07-29T13:53:19.566408+0000","last_active":"2023-07-29T14:05:36.290603+0000","last_peered":"2023-07-29T14:05:36.290603+0000","last_clean":"2023-07-29T14:05:36.290603+0000","last_became_active":"2023-07-29T13:53:19.560480+0000","last_became_peered":"2023-07-29T13:53:19.560480+0000","last_unstale":"2023-07-29T14:05:36.290603+0000","last_undegraded":"2023-07-29T14:05:36.290603+0000","last_fullsized":"2023-07-29T14:05:36.290603+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:01:49.170231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":913,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.314881+0000","last_change":"2023-07-29T13:53:19.617225+0000","last_active":"2023-07-29T14:05:36.314881+0000","last_peered":"2023-07-29T14:05:36.314881+0000","last_clean":"2023-07-29T14:05:36.314881+0000","last_became_active":"2023-07-29T13:53:19.611477+0000","last_became_peered":"2023-07-29T13:53:19.611477+0000","last_unstale":"2023-07-29T14:05:36.314881+0000","last_undegraded":"2023-07-29T14:05:36.314881+0000","last_fullsized":"2023-07-29T14:05:36.314881+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:52:56.727655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":951,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.539929+0000","last_change":"2023-07-29T13:56:55.182600+0000","last_active":"2023-07-29T14:05:37.539929+0000","last_peered":"2023-07-29T14:05:37.539929+0000","last_clean":"2023-07-29T14:05:37.539929+0000","last_became_active":"2023-07-29T13:56:55.173228+0000","last_became_peered":"2023-07-29T13:56:55.173228+0000","last_unstale":"2023-07-29T14:05:37.539929+0000","last_undegraded":"2023-07-29T14:05:37.539929+0000","last_fullsized":"2023-07-29T14:05:37.539929+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:57:08.005958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":938,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.548791+0000","last_change":"2023-07-29T13:56:55.243737+0000","last_active":"2023-07-29T14:05:37.548791+0000","last_peered":"2023-07-29T14:05:37.548791+0000","last_clean":"2023-07-29T14:05:37.548791+0000","last_became_active":"2023-07-29T13:56:55.238036+0000","last_became_peered":"2023-07-29T13:56:55.238036+0000","last_unstale":"2023-07-29T14:05:37.548791+0000","last_undegraded":"2023-07-29T14:05:37.548791+0000","last_fullsized":"2023-07-29T14:05:37.548791+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:04:27.494041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":931,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:36.745324+0000","last_change":"2023-07-29T13:29:33.527250+0000","last_active":"2023-07-29T14:05:36.745324+0000","last_peered":"2023-07-29T14:05:36.745324+0000","last_clean":"2023-07-29T14:05:36.745324+0000","last_became_active":"2023-07-29T13:29:33.508113+0000","last_became_peered":"2023-07-29T13:29:33.508113+0000","last_unstale":"2023-07-29T14:05:36.745324+0000","last_undegraded":"2023-07-29T14:05:36.745324+0000","last_fullsized":"2023-07-29T14:05:36.745324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:52:25.906655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":941,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.401769+0000","last_change":"2023-07-29T13:29:32.261353+0000","last_active":"2023-07-29T14:05:38.401769+0000","last_peered":"2023-07-29T14:05:38.401769+0000","last_clean":"2023-07-29T14:05:38.401769+0000","last_became_active":"2023-07-29T13:29:32.123788+0000","last_became_peered":"2023-07-29T13:29:32.123788+0000","last_unstale":"2023-07-29T14:05:38.401769+0000","last_undegraded":"2023-07-29T14:05:38.401769+0000","last_fullsized":"2023-07-29T14:05:38.401769+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:25:55.926424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":937,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.564990+0000","last_change":"2023-07-29T13:56:55.597264+0000","last_active":"2023-07-29T14:05:37.564990+0000","last_peered":"2023-07-29T14:05:37.564990+0000","last_clean":"2023-07-29T14:05:37.564990+0000","last_became_active":"2023-07-29T13:56:55.591631+0000","last_became_peered":"2023-07-29T13:56:55.591631+0000","last_unstale":"2023-07-29T14:05:37.564990+0000","last_undegraded":"2023-07-29T14:05:37.564990+0000","last_fullsized":"2023-07-29T14:05:37.564990+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:55:19.929908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1003,"reported_epoch":1002,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.046757+0000","last_change":"2023-07-29T13:56:55.572081+0000","last_active":"2023-07-29T14:05:37.046757+0000","last_peered":"2023-07-29T14:05:37.046757+0000","last_clean":"2023-07-29T14:05:37.046757+0000","last_became_active":"2023-07-29T13:56:55.564155+0000","last_became_peered":"2023-07-29T13:56:55.564155+0000","last_unstale":"2023-07-29T14:05:37.046757+0000","last_undegraded":"2023-07-29T14:05:37.046757+0000","last_fullsized":"2023-07-29T14:05:37.046757+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:56:05.370398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":971,"reported_epoch":1001,"state":"active+clean","last_fresh":"2023-07-29T14:05:35.554873+0000","last_change":"2023-07-29T13:53:19.441193+0000","last_active":"2023-07-29T14:05:35.554873+0000","last_peered":"2023-07-29T14:05:35.554873+0000","last_clean":"2023-07-29T14:05:35.554873+0000","last_became_active":"2023-07-29T13:53:19.433950+0000","last_became_peered":"2023-07-29T13:53:19.433950+0000","last_unstale":"2023-07-29T14:05:35.554873+0000","last_undegraded":"2023-07-29T14:05:35.554873+0000","last_fullsized":"2023-07-29T14:05:35.554873+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:38:39.350563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":950,"reported_epoch":1002,"state":"peering","last_fresh":"2023-07-29T14:05:36.299801+0000","last_change":"2023-07-29T14:05:36.303408+0000","last_active":"2023-07-29T14:05:36.299801+0000","last_peered":"2023-07-29T14:05:35.502600+0000","last_clean":"2023-07-29T14:04:06.286440+0000","last_became_active":"2023-07-29T14:05:29.629683+0000","last_became_peered":"2023-07-29T14:05:29.629683+0000","last_unstale":"2023-07-29T14:05:36.299801+0000","last_undegraded":"2023-07-29T14:05:36.299801+0000","last_fullsized":"2023-07-29T14:05:36.299801+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:52:32.253454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":17,"seq":73014444472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60040,"kb_used_data":6848,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":7012352,"data_stored":3836586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":70,"apply_latency_ms":70,"commit_latency_ns":70000000,"apply_latency_ns":70000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":15.343999999999999,"5min":16.634,"15min":170.43700000000001},"min":{"1min":6.3490000000000002,"5min":6.3330000000000002,"15min":4.7590000000000003},"max":{"1min":111.66,"5min":172.44399999999999,"15min":12751.485000000001},"last":8.1539999999999999},{"interface":"front","average":{"1min":16.254000000000001,"5min":16.774000000000001,"15min":173.22},"min":{"1min":6.79,"5min":5.4500000000000002,"15min":5.4500000000000002},"max":{"1min":129.28399999999999,"5min":173.48699999999999,"15min":12747.76},"last":15.557}]},{"osd":1,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":18.536000000000001,"5min":1543.261,"15min":525.65800000000002},"min":{"1min":7.0010000000000003,"5min":6.5149999999999997,"15min":6.5149999999999997},"max":{"1min":99.557000000000002,"5min":29038.089,"15min":29038.089},"last":58.942},{"interface":"front","average":{"1min":22.518000000000001,"5min":1548.174,"15min":530.21500000000003},"min":{"1min":6.9089999999999998,"5min":6.4749999999999996,"15min":6.2249999999999996},"max":{"1min":102.327,"5min":29039.032999999999,"15min":29039.032999999999},"last":53.643999999999998}]},{"osd":2,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":23.297000000000001,"5min":21.439,"15min":20.452000000000002},"min":{"1min":7.992,"5min":7.3200000000000003,"15min":7.3200000000000003},"max":{"1min":104.566,"5min":249.84800000000001,"15min":784.66200000000003},"last":21.434000000000001},{"interface":"front","average":{"1min":21.670000000000002,"5min":21.084,"15min":19.474},"min":{"1min":8.6280000000000001,"5min":8.3919999999999995,"15min":7.8049999999999997},"max":{"1min":99.706000000000003,"5min":202.64500000000001,"15min":368.62200000000001},"last":25.123999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.91,"5min":25.608000000000001,"15min":478.83800000000002},"min":{"1min":7.0910000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"max":{"1min":101.714,"5min":312.73599999999999,"15min":28381.707999999999},"last":22.844000000000001},{"interface":"front","average":{"1min":20.298999999999999,"5min":27.803999999999998,"15min":474.06299999999999},"min":{"1min":9.3219999999999992,"5min":9.3219999999999992,"15min":8.7520000000000007},"max":{"1min":97.727999999999994,"5min":242.63,"15min":28126.012999999999},"last":25.978000000000002}]},{"osd":5,"last update":"Sat Jul 29 14:05:01 2023","interfaces":[{"interface":"back","average":{"1min":17.222999999999999,"5min":20.141999999999999,"15min":32.993000000000002},"min":{"1min":10.452,"5min":8.4540000000000006,"15min":7.3689999999999998},"max":{"1min":79.867000000000004,"5min":85.578999999999994,"15min":488.72300000000001},"last":58.226999999999997},{"interface":"front","average":{"1min":14.161,"5min":19.236999999999998,"15min":21.838000000000001},"min":{"1min":8.1379999999999999,"5min":8.1379999999999999,"15min":7.3840000000000003},"max":{"1min":52.622,"5min":106.634,"15min":128.791},"last":51.494}]},{"osd":6,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":24.904,"5min":20.061,"15min":22.629999999999999},"min":{"1min":9.1470000000000002,"5min":8.0129999999999999,"15min":7.7779999999999996},"max":{"1min":106.33199999999999,"5min":124.819,"15min":787.18499999999995},"last":54.243000000000002},{"interface":"front","average":{"1min":25.198,"5min":20.042000000000002,"15min":22.454999999999998},"min":{"1min":9.8010000000000002,"5min":8.5790000000000006,"15min":8.3170000000000002},"max":{"1min":112.328,"5min":112.328,"15min":646.87599999999998},"last":35.186999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":27.215,"5min":22.292000000000002,"15min":48.816000000000003},"min":{"1min":11.487,"5min":9.2620000000000005,"15min":9.2620000000000005},"max":{"1min":149.43199999999999,"5min":149.43199999999999,"15min":648.54200000000003},"last":36.865000000000002},{"interface":"front","average":{"1min":23.321000000000002,"5min":20.568999999999999,"15min":38.241999999999997},"min":{"1min":10.385,"5min":9.2300000000000004,"15min":8.7309999999999999},"max":{"1min":75.954999999999998,"5min":75.954999999999998,"15min":364.86500000000001},"last":30.257999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444484,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64112,"kb_used_data":6824,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94307728,"statfs":{"total":96636764160,"available":96571113472,"internally_reserved":0,"allocated":6987776,"data_stored":3804439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":13.247999999999999,"15min":130.82599999999999},"min":{"1min":5.3789999999999996,"5min":5.0209999999999999,"15min":4.476},"max":{"1min":29.634,"5min":93.590999999999994,"15min":10009.493},"last":12.676},{"interface":"front","average":{"1min":13.936999999999999,"5min":13.526,"15min":131.76599999999999},"min":{"1min":7.1299999999999999,"5min":6.2069999999999999,"15min":5.617},"max":{"1min":37.527999999999999,"5min":99.156999999999996,"15min":9995.5869999999995},"last":10.468}]},{"osd":1,"last update":"Sat Jul 29 14:04:47 2023","interfaces":[{"interface":"back","average":{"1min":11.255000000000001,"5min":1413.5239999999999,"15min":481.31099999999998},"min":{"1min":6.0899999999999999,"5min":6.0899999999999999,"15min":6.0899999999999999},"max":{"1min":17.850999999999999,"5min":29343.018,"15min":29343.018},"last":101.836},{"interface":"front","average":{"1min":11.234999999999999,"5min":1411.289,"15min":478.77199999999999},"min":{"1min":7.0209999999999999,"5min":6.5880000000000001,"15min":5.8860000000000001},"max":{"1min":18.347000000000001,"5min":29342.263999999999,"15min":29342.263999999999},"last":137.94}]},{"osd":3,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":17.623000000000001,"5min":20.777000000000001,"15min":365.04399999999998},"min":{"1min":6.7809999999999997,"5min":6.7809999999999997,"15min":6.7809999999999997},"max":{"1min":86.926000000000002,"5min":331.92399999999998,"15min":29770.313999999998},"last":11.025},{"interface":"front","average":{"1min":21.684999999999999,"5min":22.039999999999999,"15min":358.54500000000002},"min":{"1min":9.3070000000000004,"5min":8.0540000000000003,"15min":6.4749999999999996},"max":{"1min":89.156000000000006,"5min":332.49900000000002,"15min":29586.387999999999},"last":7.351}]},{"osd":4,"last update":"Sat Jul 29 14:04:53 2023","interfaces":[{"interface":"back","average":{"1min":12.471,"5min":819.34000000000003,"15min":870.13199999999995},"min":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"max":{"1min":52.584000000000003,"5min":25685.629000000001,"15min":32071.659},"last":6.5810000000000004},{"interface":"front","average":{"1min":15.125999999999999,"5min":814.09699999999998,"15min":870.976},"min":{"1min":8.8330000000000002,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":35.020000000000003,"5min":25610.863000000001,"15min":32126.150000000001},"last":8.0410000000000004}]},{"osd":5,"last update":"Sat Jul 29 14:05:06 2023","interfaces":[{"interface":"back","average":{"1min":14.398,"5min":20.428999999999998,"15min":38.106999999999999},"min":{"1min":8.7579999999999991,"5min":8.4269999999999996,"15min":8.4269999999999996},"max":{"1min":18.667000000000002,"5min":193.45400000000001,"15min":428.04000000000002},"last":11.632999999999999},{"interface":"front","average":{"1min":12.019,"5min":20.446999999999999,"15min":41.822000000000003},"min":{"1min":6.891,"5min":6.891,"15min":6.891},"max":{"1min":19.634,"5min":174.292,"15min":430.15499999999997},"last":10.163}]},{"osd":6,"last update":"Sat Jul 29 14:04:38 2023","interfaces":[{"interface":"back","average":{"1min":14.581,"5min":23.431000000000001,"15min":19.853000000000002},"min":{"1min":8.5359999999999996,"5min":8.5359999999999996,"15min":8.5359999999999996},"max":{"1min":24.651,"5min":318.45299999999997,"15min":318.45299999999997},"last":12.380000000000001},{"interface":"front","average":{"1min":15.903,"5min":21.638000000000002,"15min":20.547999999999998},"min":{"1min":11.417999999999999,"5min":9.7789999999999999,"15min":7.5300000000000002},"max":{"1min":22.896999999999998,"5min":349.34899999999999,"15min":349.34899999999999},"last":9.2230000000000008}]},{"osd":7,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":15.279999999999999,"5min":20.536000000000001,"15min":21.027000000000001},"min":{"1min":10.663,"5min":8.5079999999999991,"15min":7.5890000000000004},"max":{"1min":20.757999999999999,"5min":291.80900000000003,"15min":291.80900000000003},"last":25.786000000000001},{"interface":"front","average":{"1min":15.644,"5min":21.513000000000002,"15min":22.518000000000001},"min":{"1min":9.6419999999999995,"5min":9.1940000000000008,"15min":9.1940000000000008},"max":{"1min":23.408000000000001,"5min":291.44200000000001,"15min":291.44200000000001},"last":82.373999999999995}]}]},{"osd":1,"up_from":447,"seq":1919850381742,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60408,"kb_used_data":7216,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311432,"statfs":{"total":96636764160,"available":96574906368,"internally_reserved":0,"allocated":7389184,"data_stored":4202279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":20.931000000000001,"5min":20.015999999999998,"15min":112.807},"min":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.3680000000000003},"max":{"1min":116.512,"5min":364.47800000000001,"15min":12525.57},"last":10.975},{"interface":"front","average":{"1min":18.943000000000001,"5min":20.664000000000001,"15min":112.874},"min":{"1min":6.0970000000000004,"5min":6.0970000000000004,"15min":4.6820000000000004},"max":{"1min":100.402,"5min":372.00999999999999,"15min":12523.974},"last":8.8849999999999998}]},{"osd":2,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.987,"5min":26.809000000000001,"15min":19.399999999999999},"min":{"1min":7.819,"5min":6.5110000000000001,"15min":5.5919999999999996},"max":{"1min":69.497,"5min":360.88099999999997,"15min":360.88099999999997},"last":8.4369999999999994},{"interface":"front","average":{"1min":19.43,"5min":28.155999999999999,"15min":20.445},"min":{"1min":7.093,"5min":7.093,"15min":6.2729999999999997},"max":{"1min":93.792000000000002,"5min":368.51400000000001,"15min":368.51400000000001},"last":7.2009999999999996}]},{"osd":3,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":26.696000000000002,"5min":30.559000000000001,"15min":531.69799999999998},"min":{"1min":9.1590000000000007,"5min":6.7140000000000004,"15min":6.7140000000000004},"max":{"1min":216.16300000000001,"5min":435.92200000000003,"15min":34504.942000000003},"last":18.053000000000001},{"interface":"front","average":{"1min":31.332000000000001,"5min":41.100999999999999,"15min":539.97299999999996},"min":{"1min":10.423,"5min":10.423,"15min":5.3639999999999999},"max":{"1min":238.33699999999999,"5min":700.10500000000002,"15min":34620.970999999998},"last":29.898}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.427,"5min":1170.991,"15min":980.98599999999999},"min":{"1min":8.4130000000000003,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":50.289000000000001,"5min":25247.098999999998,"15min":32258.529999999999},"last":11.036},{"interface":"front","average":{"1min":16.536000000000001,"5min":1165.587,"15min":967.39999999999998},"min":{"1min":11.340999999999999,"5min":8.9169999999999998,"15min":6.5090000000000003},"max":{"1min":29.376999999999999,"5min":25174.008999999998,"15min":31933.992999999999},"last":9.3900000000000006}]},{"osd":5,"last update":"Sat Jul 29 14:05:13 2023","interfaces":[{"interface":"back","average":{"1min":15.249000000000001,"5min":33.526000000000003,"15min":43.616},"min":{"1min":10.416,"5min":10.294,"15min":10.294},"max":{"1min":23.649999999999999,"5min":433.79300000000001,"15min":433.79300000000001},"last":81.477999999999994},{"interface":"front","average":{"1min":15.952,"5min":36.515000000000001,"15min":46.286000000000001},"min":{"1min":11.467000000000001,"5min":9.6929999999999996,"15min":8.7260000000000009},"max":{"1min":22.300999999999998,"5min":419.22800000000001,"15min":419.22800000000001},"last":88.980999999999995}]},{"osd":6,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":16.931999999999999,"5min":21.992000000000001,"15min":19.459},"min":{"1min":6.6079999999999997,"5min":6.327,"15min":6.327},"max":{"1min":154.92400000000001,"5min":357.20499999999998,"15min":357.20499999999998},"last":15.124000000000001},{"interface":"front","average":{"1min":20.251999999999999,"5min":27.460999999999999,"15min":23.411000000000001},"min":{"1min":10.877000000000001,"5min":9.5079999999999991,"15min":9.5079999999999991},"max":{"1min":155.994,"5min":359.68099999999998,"15min":359.68099999999998},"last":14.048}]},{"osd":7,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.683,"5min":35.137999999999998,"15min":23.725999999999999},"min":{"1min":10.130000000000001,"5min":8.5199999999999996,"15min":7.5919999999999996},"max":{"1min":92.168999999999997,"5min":563.29700000000003,"15min":563.29700000000003},"last":12.036},{"interface":"front","average":{"1min":17.949000000000002,"5min":30.442,"15min":21.349},"min":{"1min":8.2240000000000002,"5min":8.2240000000000002,"15min":7.2919999999999998},"max":{"1min":113.15900000000001,"5min":352.65199999999999,"15min":352.65199999999999},"last":9.9580000000000002}]}]},{"osd":0,"up_from":1002,"seq":4303557231028,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60024,"kb_used_data":6472,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311816,"statfs":{"total":96636764160,"available":96575299584,"internally_reserved":0,"allocated":6627328,"data_stored":3587977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":77,"apply_latency_ms":77,"commit_latency_ns":77000000,"apply_latency_ns":77000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":71.099999999999994,"5min":1023.201,"15min":351.10899999999998},"min":{"1min":8.9689999999999994,"5min":5.9249999999999998,"15min":4.6269999999999998},"max":{"1min":350.00200000000001,"5min":27534,"15min":27534},"last":6.1740000000000004},{"interface":"front","average":{"1min":99.875,"5min":1025.7080000000001,"15min":350.98899999999998},"min":{"1min":5.9589999999999996,"5min":5.7240000000000002,"15min":5.0739999999999998},"max":{"1min":591.69600000000003,"5min":27535.008999999998,"15min":27535.008999999998},"last":6.6829999999999998}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.170999999999999,"5min":22.285,"15min":18.638000000000002},"min":{"1min":11.680999999999999,"5min":8.1069999999999993,"15min":6.6159999999999997},"max":{"1min":364.62799999999999,"5min":364.62799999999999,"15min":364.62799999999999},"last":9.9009999999999998},{"interface":"front","average":{"1min":45.890000000000001,"5min":26.417999999999999,"15min":21.443000000000001},"min":{"1min":14.172000000000001,"5min":10.023999999999999,"15min":8.8780000000000001},"max":{"1min":370.34300000000002,"5min":370.34300000000002,"15min":370.34300000000002},"last":11.736000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":66.832999999999998,"5min":25.178000000000001,"15min":355.89100000000002},"min":{"1min":6.7610000000000001,"5min":6.4409999999999998,"15min":5.891},"max":{"1min":444.59500000000003,"5min":444.59500000000003,"15min":30170.040000000001},"last":96.820999999999998},{"interface":"front","average":{"1min":65.754000000000005,"5min":33.606000000000002,"15min":363.07400000000001},"min":{"1min":10.266999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"max":{"1min":386.06099999999998,"5min":386.06099999999998,"15min":30171.194},"last":74.893000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":92.555000000000007,"5min":36.014000000000003,"15min":585.43899999999996},"min":{"1min":12.201000000000001,"5min":6.4480000000000004,"15min":6.0979999999999999},"max":{"1min":445.65300000000002,"5min":445.65300000000002,"15min":29892.669999999998},"last":11.266},{"interface":"front","average":{"1min":115.806,"5min":40.560000000000002,"15min":588.88900000000001},"min":{"1min":7.923,"5min":5.9400000000000004,"15min":5.9400000000000004},"max":{"1min":593.13499999999999,"5min":593.13499999999999,"15min":29877.289000000001},"last":14.489000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.341000000000001,"5min":28.972000000000001,"15min":56.932000000000002},"min":{"1min":12.702,"5min":9.5700000000000003,"15min":9.5700000000000003},"max":{"1min":403.13999999999999,"5min":403.13999999999999,"15min":508.02100000000002},"last":94.206999999999994},{"interface":"front","average":{"1min":43.966999999999999,"5min":23.216000000000001,"15min":32.374000000000002},"min":{"1min":7.6429999999999998,"5min":7.1210000000000004,"15min":7.1210000000000004},"max":{"1min":590.58500000000004,"5min":590.58500000000004,"15min":590.58500000000004},"last":92.817999999999998}]},{"osd":6,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":52.365000000000002,"5min":30.853999999999999,"15min":25.227},"min":{"1min":16.876000000000001,"5min":10.409000000000001,"15min":9.9350000000000005},"max":{"1min":375.36500000000001,"5min":375.36500000000001,"15min":375.36500000000001},"last":51.311},{"interface":"front","average":{"1min":55.634,"5min":32.579999999999998,"15min":27.417999999999999},"min":{"1min":18.984999999999999,"5min":11.564,"15min":11.564},"max":{"1min":380.31599999999997,"5min":380.31599999999997,"15min":380.31599999999997},"last":71.132999999999996}]},{"osd":7,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":57.960999999999999,"5min":28.638000000000002,"15min":20.780000000000001},"min":{"1min":17.748999999999999,"5min":8.4939999999999998,"15min":7.9349999999999996},"max":{"1min":385.01900000000001,"5min":385.01900000000001,"15min":385.01900000000001},"last":13.416},{"interface":"front","average":{"1min":54.853999999999999,"5min":29.765999999999998,"15min":25.545000000000002},"min":{"1min":10.992000000000001,"5min":8.1229999999999993,"15min":8.1229999999999993},"max":{"1min":446.69099999999997,"5min":446.69099999999997,"15min":446.69099999999997},"last":15.956}]}]},{"osd":4,"up_from":16,"seq":68719477161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60104,"kb_used_data":6912,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311736,"statfs":{"total":96636764160,"available":96575217664,"internally_reserved":0,"allocated":7077888,"data_stored":3907142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":13.432,"5min":21.457000000000001,"15min":87.253},"min":{"1min":6.4029999999999996,"5min":4.6230000000000002,"15min":4.6230000000000002},"max":{"1min":48.048999999999999,"5min":495.86900000000003,"15min":10627.831},"last":11.238},{"interface":"front","average":{"1min":18.010000000000002,"5min":25.013000000000002,"15min":90.852000000000004},"min":{"1min":7.2400000000000002,"5min":6.0990000000000002,"15min":5.6660000000000004},"max":{"1min":124.64100000000001,"5min":446.84100000000001,"15min":10653.877},"last":7.7329999999999997}]},{"osd":1,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":11.276999999999999,"5min":1441.567,"15min":495.98200000000003},"min":{"1min":7.2640000000000002,"5min":7.2640000000000002,"15min":5.9909999999999997},"max":{"1min":21.369,"5min":26872.857,"15min":26872.857},"last":8.4139999999999997},{"interface":"front","average":{"1min":11.013,"5min":1440.0119999999999,"15min":493.40499999999997},"min":{"1min":7.4269999999999996,"5min":7.3090000000000002,"15min":5.6909999999999998},"max":{"1min":17.440999999999999,"5min":26874.048999999999,"15min":26874.048999999999},"last":10.928000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":12.445,"5min":24.050999999999998,"15min":23.050000000000001},"min":{"1min":8.9000000000000004,"5min":7.3019999999999996,"15min":7.3019999999999996},"max":{"1min":15.904999999999999,"5min":453.40100000000001,"15min":597.11800000000005},"last":17.305},{"interface":"front","average":{"1min":15.029999999999999,"5min":27.077999999999999,"15min":26.128},"min":{"1min":12.429,"5min":11.631,"15min":8.0559999999999992},"max":{"1min":19.111000000000001,"5min":453.96199999999999,"15min":705.04499999999996},"last":57.628}]},{"osd":3,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":14.789999999999999,"5min":27.626999999999999,"15min":590.51199999999994},"min":{"1min":9.7219999999999995,"5min":7.6799999999999997,"15min":5.9740000000000002},"max":{"1min":23.942,"5min":448.65100000000001,"15min":30725.612000000001},"last":13.273999999999999},{"interface":"front","average":{"1min":12.497,"5min":20.047999999999998,"15min":582.06899999999996},"min":{"1min":7.7690000000000001,"5min":6.3840000000000003,"15min":6.3840000000000003},"max":{"1min":19.815000000000001,"5min":336.62599999999998,"15min":30407.659},"last":15.597}]},{"osd":5,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":16.190999999999999,"5min":30.579000000000001,"15min":32.869},"min":{"1min":14.1,"5min":7.1440000000000001,"15min":7.1440000000000001},"max":{"1min":18.738,"5min":450.81700000000001,"15min":450.81700000000001},"last":14.425000000000001},{"interface":"front","average":{"1min":10.430999999999999,"5min":19.959,"15min":22.803999999999998},"min":{"1min":6.2779999999999996,"5min":6.2779999999999996,"15min":6.2779999999999996},"max":{"1min":20.254999999999999,"5min":340.04000000000002,"15min":340.04000000000002},"last":7.9279999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":13.007,"5min":22.312000000000001,"15min":22.866},"min":{"1min":8.1120000000000001,"5min":7.1689999999999996,"15min":6.1520000000000001},"max":{"1min":22.010000000000002,"5min":339.47199999999998,"15min":696.952},"last":10.199999999999999},{"interface":"front","average":{"1min":14.724,"5min":23.917999999999999,"15min":23.209},"min":{"1min":9.0289999999999999,"5min":6.3449999999999998,"15min":6.3449999999999998},"max":{"1min":20.968,"5min":498.26999999999998,"15min":598.86900000000003},"last":16.390999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":12.805,"5min":24.097000000000001,"15min":24.802},"min":{"1min":8.7430000000000003,"5min":8.3399999999999999,"15min":7.7930000000000001},"max":{"1min":18.638000000000002,"5min":340.61900000000003,"15min":699.31799999999998},"last":13.837999999999999},{"interface":"front","average":{"1min":17.483000000000001,"5min":31.318999999999999,"15min":33.040999999999997},"min":{"1min":13.861000000000001,"5min":11.067,"15min":11.067},"max":{"1min":19.817,"5min":455.339,"15min":703.68200000000002},"last":11.691000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509874,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82180,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94289660,"statfs":{"total":96636764160,"available":96552611840,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":12.891999999999999,"5min":16.984000000000002,"15min":62.423000000000002},"min":{"1min":6.9390000000000001,"5min":5.2350000000000003,"15min":5.2350000000000003},"max":{"1min":29.821000000000002,"5min":224.93899999999999,"15min":9502.5869999999995},"last":5.5629999999999997},{"interface":"front","average":{"1min":15.925000000000001,"5min":19.867000000000001,"15min":65.921999999999997},"min":{"1min":8.2050000000000001,"5min":6.1580000000000004,"15min":6.1580000000000004},"max":{"1min":31.303999999999998,"5min":130.48400000000001,"15min":9511.3719999999994},"last":9.6059999999999999}]},{"osd":1,"last update":"Sat Jul 29 14:04:49 2023","interfaces":[{"interface":"back","average":{"1min":17.140999999999998,"5min":970.20899999999995,"15min":335.46300000000002},"min":{"1min":7.5839999999999996,"5min":7.3369999999999997,"15min":4.8529999999999998},"max":{"1min":73.679000000000002,"5min":26903.921999999999,"15min":26903.921999999999},"last":14.039999999999999},{"interface":"front","average":{"1min":12.878,"5min":963.24699999999996,"15min":332.12599999999998},"min":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.3330000000000002},"max":{"1min":69.432000000000002,"5min":26893.022000000001,"15min":26893.022000000001},"last":11.784000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":14.891999999999999,"5min":21.295999999999999,"15min":19.027000000000001},"min":{"1min":10.173,"5min":7.8609999999999998,"15min":6.6299999999999999},"max":{"1min":26.302,"5min":191.73099999999999,"15min":191.73099999999999},"last":15.919},{"interface":"front","average":{"1min":11.848000000000001,"5min":16.003,"15min":15.422000000000001},"min":{"1min":7.4690000000000003,"5min":5.5499999999999998,"15min":5.5499999999999998},"max":{"1min":18,"5min":189.40199999999999,"15min":189.40199999999999},"last":10.874000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":10.653,"5min":15.525,"15min":521.48800000000006},"min":{"1min":7.2619999999999996,"5min":6.335,"15min":6.335},"max":{"1min":23.292000000000002,"5min":187.852,"15min":30710.619999999999},"last":12.523999999999999},{"interface":"front","average":{"1min":15.202999999999999,"5min":17.899000000000001,"15min":521.02300000000002},"min":{"1min":9.907,"5min":7.7009999999999996,"15min":7.6180000000000003},"max":{"1min":40.335000000000001,"5min":101.88200000000001,"15min":30591.776999999998},"last":17.108000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:04:56 2023","interfaces":[{"interface":"back","average":{"1min":16.030000000000001,"5min":808.61300000000006,"15min":1019.547},"min":{"1min":8.0950000000000006,"5min":6.718,"15min":6.1070000000000002},"max":{"1min":73.055999999999997,"5min":25620.438999999998,"15min":32899.582000000002},"last":8.8569999999999993},{"interface":"front","average":{"1min":16.128,"5min":828.35799999999995,"15min":1009.853},"min":{"1min":10.254,"5min":6.2089999999999996,"15min":6.2089999999999996},"max":{"1min":40.786999999999999,"5min":25979.712,"15min":32485.278999999999},"last":15.188000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:05:04 2023","interfaces":[{"interface":"back","average":{"1min":17.489999999999998,"5min":21.376999999999999,"15min":24.847000000000001},"min":{"1min":11.52,"5min":9.4949999999999992,"15min":8.9290000000000003},"max":{"1min":34.677,"5min":75.25,"15min":189.423},"last":53.837000000000003},{"interface":"front","average":{"1min":14.324,"5min":18.073,"15min":21.690000000000001},"min":{"1min":8.6969999999999992,"5min":7.3330000000000002,"15min":7.3330000000000002},"max":{"1min":24.77,"5min":80.085999999999999,"15min":188.518},"last":52.890000000000001}]},{"osd":7,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":24.321000000000002,"5min":19.489999999999998,"15min":19.814},"min":{"1min":14.164,"5min":7.1059999999999999,"15min":7.1059999999999999},"max":{"1min":74.337000000000003,"5min":74.337000000000003,"15min":200.55099999999999},"last":19.123000000000001},{"interface":"front","average":{"1min":18.581,"5min":20.780000000000001,"15min":22.265000000000001},"min":{"1min":9.6880000000000006,"5min":8.0790000000000006,"15min":8.0790000000000006},"max":{"1min":47.313000000000002,"5min":89.125,"15min":222.65199999999999},"last":17.780999999999999}]}]},{"osd":5,"up_from":704,"seq":3023656976773,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78476,"kb_used_data":6636,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94293364,"statfs":{"total":96636764160,"available":96556404736,"internally_reserved":0,"allocated":6795264,"data_stored":3844748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:04:00 2023","interfaces":[{"interface":"back","average":{"1min":23.48,"5min":22.623000000000001,"15min":41.000999999999998},"min":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":4.742},"max":{"1min":150.63,"5min":262.56099999999998,"15min":299.87799999999999},"last":8.3529999999999998},{"interface":"front","average":{"1min":24.837,"5min":22.727,"15min":52.545999999999999},"min":{"1min":5.8259999999999996,"5min":4.6539999999999999,"15min":4.6539999999999999},"max":{"1min":111.14100000000001,"5min":253.80000000000001,"15min":370.00999999999999},"last":7.3300000000000001}]},{"osd":1,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":13.867000000000001,"5min":25.277000000000001,"15min":33.274999999999999},"min":{"1min":6.5419999999999998,"5min":5.3780000000000001,"15min":5.3780000000000001},"max":{"1min":91.590000000000003,"5min":362.58800000000002,"15min":362.58800000000002},"last":12.525},{"interface":"front","average":{"1min":15.465999999999999,"5min":34.079000000000001,"15min":56.872999999999998},"min":{"1min":7.5650000000000004,"5min":6.8680000000000003,"15min":6.8680000000000003},"max":{"1min":85.760000000000005,"5min":356.69900000000001,"15min":363.50999999999999},"last":9.1530000000000005}]},{"osd":2,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.152000000000001,"5min":26.193000000000001,"15min":51.703000000000003},"min":{"1min":8.1880000000000006,"5min":7.032,"15min":7.032},"max":{"1min":151.804,"5min":312.755,"15min":384.024},"last":18.709},{"interface":"front","average":{"1min":16.754000000000001,"5min":20.221,"15min":25.251000000000001},"min":{"1min":7.782,"5min":7.2759999999999998,"15min":7.2709999999999999},"max":{"1min":102.09099999999999,"5min":228.55600000000001,"15min":228.55600000000001},"last":14.336}]},{"osd":3,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":28.802,"15min":55.957999999999998},"min":{"1min":10.734,"5min":7.4100000000000001,"15min":7.4100000000000001},"max":{"1min":131.36600000000001,"5min":312.12200000000001,"15min":381.23200000000003},"last":14.715},{"interface":"front","average":{"1min":18.568999999999999,"5min":28.875,"15min":66.894999999999996},"min":{"1min":7.944,"5min":7.944,"15min":7.944},"max":{"1min":118.456,"5min":265.92700000000002,"15min":382.779},"last":13.188000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":19.385999999999999,"5min":1108.5630000000001,"15min":412.19600000000003},"min":{"1min":10.5,"5min":7.343,"15min":7.3070000000000004},"max":{"1min":103.768,"5min":23572.741999999998,"15min":23572.741999999998},"last":16.469999999999999},{"interface":"front","average":{"1min":20.039000000000001,"5min":1113.885,"15min":428.73500000000001},"min":{"1min":8.7260000000000009,"5min":6.8929999999999998,"15min":6.8929999999999998},"max":{"1min":137.57400000000001,"5min":23610.142,"15min":23610.142},"last":9.8550000000000004}]},{"osd":6,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":17.074000000000002,"5min":25.704999999999998,"15min":51.972000000000001},"min":{"1min":9.7360000000000007,"5min":8.1709999999999994,"15min":8.1709999999999994},"max":{"1min":41.347000000000001,"5min":263.983,"15min":450.17500000000001},"last":11.878},{"interface":"front","average":{"1min":18.436,"5min":26.684999999999999,"15min":57.575000000000003},"min":{"1min":11.185,"5min":8.1969999999999992,"15min":8.1969999999999992},"max":{"1min":63.707999999999998,"5min":221.99199999999999,"15min":417.303},"last":17.875}]},{"osd":7,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":21.550999999999998,"5min":28.064,"15min":47.530999999999999},"min":{"1min":12.340999999999999,"5min":11.222,"15min":10.602},"max":{"1min":104.718,"5min":284.48000000000002,"15min":341.30500000000001},"last":19.625},{"interface":"front","average":{"1min":18.460999999999999,"5min":27.806000000000001,"15min":53.993000000000002},"min":{"1min":12.089,"5min":11.042999999999999,"15min":11.042999999999999},"max":{"1min":46.353999999999999,"5min":191.33099999999999,"15min":329.86500000000001},"last":15.612}]}]},{"osd":7,"up_from":650,"seq":2791728742800,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60024,"kb_used_data":6472,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311816,"statfs":{"total":96636764160,"available":96575299584,"internally_reserved":0,"allocated":6627328,"data_stored":3587977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:45 2023","interfaces":[{"interface":"back","average":{"1min":11.94,"5min":13.635,"15min":23.015000000000001},"min":{"1min":5.5890000000000004,"5min":5.4539999999999997,"15min":5.4539999999999997},"max":{"1min":28.5,"5min":75.968000000000004,"15min":293.94499999999999},"last":11.478999999999999},{"interface":"front","average":{"1min":14.968999999999999,"5min":18.658999999999999,"15min":28.620000000000001},"min":{"1min":6.8179999999999996,"5min":6.8179999999999996,"15min":6.8179999999999996},"max":{"1min":29.143999999999998,"5min":116.952,"15min":358.267},"last":7.9210000000000003}]},{"osd":1,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":19.048999999999999,"5min":1108.855,"15min":389.67099999999999},"min":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":4.6660000000000004},"max":{"1min":184.25800000000001,"5min":27546.963,"15min":27546.963},"last":12.223000000000001},{"interface":"front","average":{"1min":22.969999999999999,"5min":1111.8620000000001,"15min":385.10300000000001},"min":{"1min":8.6110000000000007,"5min":5.5010000000000003,"15min":5.5010000000000003},"max":{"1min":150.75700000000001,"5min":27546.280999999999,"15min":27546.280999999999},"last":14.646000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.088000000000001,"5min":19.381,"15min":28.02},"min":{"1min":10.154999999999999,"5min":7.9100000000000001,"15min":7.9100000000000001},"max":{"1min":103.09,"5min":103.09,"15min":461.83800000000002},"last":12.978999999999999},{"interface":"front","average":{"1min":21.068000000000001,"5min":15.704000000000001,"15min":23.861999999999998},"min":{"1min":7.1269999999999998,"5min":6.6239999999999997,"15min":6.6239999999999997},"max":{"1min":181.97900000000001,"5min":181.97900000000001,"15min":385.40800000000002},"last":8.9149999999999991}]},{"osd":3,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":25.065999999999999,"5min":21.029,"15min":1531.4280000000001},"min":{"1min":7.976,"5min":7.6660000000000004,"15min":7.6660000000000004},"max":{"1min":242.197,"5min":242.197,"15min":34550.838000000003},"last":16.02},{"interface":"front","average":{"1min":31.428999999999998,"5min":27.873000000000001,"15min":1543.2929999999999},"min":{"1min":12.592000000000001,"5min":6.9550000000000001,"15min":6.9550000000000001},"max":{"1min":240.46899999999999,"5min":243.06399999999999,"15min":34556.381000000001},"last":18.369}]},{"osd":4,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":20.402999999999999,"5min":1074.817,"15min":900.84500000000003},"min":{"1min":6.7220000000000004,"5min":6.3719999999999999,"15min":6.3719999999999999},"max":{"1min":103.931,"5min":25146.41,"15min":27891.68},"last":9.7100000000000009},{"interface":"front","average":{"1min":29.045999999999999,"5min":1099.6300000000001,"15min":918.40200000000004},"min":{"1min":9.7360000000000007,"5min":8.1630000000000003,"15min":7.2699999999999996},"max":{"1min":189.47900000000001,"5min":25434.151999999998,"15min":27883.419999999998},"last":21.300999999999998}]},{"osd":5,"last update":"Sat Jul 29 14:05:08 2023","interfaces":[{"interface":"back","average":{"1min":17.529,"5min":28.314,"15min":72.040999999999997},"min":{"1min":14.367000000000001,"5min":11.215,"15min":10.683},"max":{"1min":22.012,"5min":230.16800000000001,"15min":1265.123},"last":21.904},{"interface":"front","average":{"1min":12.343999999999999,"5min":25.687999999999999,"15min":76.659000000000006},"min":{"1min":9.609,"5min":7.71,"15min":7.5540000000000003},"max":{"1min":21.030999999999999,"5min":255.23699999999999,"15min":1234.9300000000001},"last":15.369999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":21.149999999999999,"15min":25.884},"min":{"1min":10.222,"5min":9.9740000000000002,"15min":7.8689999999999998},"max":{"1min":78.843000000000004,"5min":95.558999999999997,"15min":243.66499999999999},"last":16.655000000000001},{"interface":"front","average":{"1min":24.378,"5min":25.236999999999998,"15min":32.558},"min":{"1min":12.111000000000001,"5min":10.542,"15min":10.542},"max":{"1min":92.266999999999996,"5min":148.334,"15min":454.68799999999999},"last":19.009}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-29T14:05:45.127 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-29T14:05:45.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.238+0000 7f69cac4f640 10 monclient: get_monmap_and_config 2023-07-29T14:05:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.238+0000 7f69cac4f640 10 monclient: build_initial_monmap 2023-07-29T14:05:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.238+0000 7f69cac4f640 10 monclient: monmap: 2023-07-29T14:05:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:45.239298+0000 2023-07-29T14:05:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:45.239298+0000 2023-07-29T14:05:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:45.244 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.238+0000 7f69cac4f640 1 Processor -- start 2023-07-29T14:05:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.238+0000 7f69cac4f640 1 -- start start 2023-07-29T14:05:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.238+0000 7f69cac4f640 10 monclient: init 2023-07-29T14:05:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2023-07-29T14:05:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f69c40b74a0 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f69c413e4f0 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69c40aa150 0x7f69c4144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 10 monclient(hunting): picked mon.noname-a con 0x7f69c40aa150 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c40b7870 con 0x7f69c413e4f0 2023-07-29T14:05:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c4145230 con 0x7f69c40b74a0 2023-07-29T14:05:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c40a91b0 con 0x7f69c40aa150 2023-07-29T14:05:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.239+0000 7f69cac4f640 10 monclient(hunting): authenticate will time out at 3667.372638s 2023-07-29T14:05:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.240+0000 7f69c91ec640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69c40aa150 0x7f69c4144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.240+0000 7f69c3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f69c413e4f0 0x7f69c41445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51742/0 (socket says 172.21.15.40:51742) 2023-07-29T14:05:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.240+0000 7f69c3fff640 1 -- 172.21.15.40:0/3173878169 learned_addr learned my addr 172.21.15.40:0/3173878169 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.240+0000 7f69c91ec640 10 monclient(hunting): get_auth_request con 0x7f69c40aa150 auth_method 0 2023-07-29T14:05:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.240+0000 7f69c91ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.240+0000 7f69c91ec640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.240+0000 7f69c91ec640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.242+0000 7f69c91ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.242+0000 7f69c91ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.242+0000 7f69c91ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.244+0000 7f69c37fe640 1 -- 172.21.15.40:0/3173878169 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76839694 0 0) 0x7f69c4145230 con 0x7f69c40b74a0 2023-07-29T14:05:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.244+0000 7f69c37fe640 10 monclient(hunting): _negotiate 2023-07-29T14:05:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.244+0000 7f69c37fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.244+0000 7f69c37fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.244+0000 7f69c37fe640 10 monclient(hunting): my global_id is 20332 2023-07-29T14:05:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.244+0000 7f69c37fe640 1 -- 172.21.15.40:0/3173878169 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69ac003520 con 0x7f69c40b74a0 2023-07-29T14:05:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.245+0000 7f69c91ec640 10 monclient(hunting): handle_auth_done global_id 14883 payload 274 2023-07-29T14:05:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.245+0000 7f69c91ec640 1 -- 172.21.15.40:0/3173878169 >> v1:172.21.15.45:6789/0 conn(0x7f69c40b74a0 legacy=0x7f69c413dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.245+0000 7f69c91ec640 1 -- 172.21.15.40:0/3173878169 >> v1:172.21.15.40:6789/0 conn(0x7f69c413e4f0 legacy=0x7f69c41445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.245+0000 7f69c91ec640 10 monclient: _finish_hunting 0 2023-07-29T14:05:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.245+0000 7f69c91ec640 1 monclient: found mon.noname-a 2023-07-29T14:05:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.245+0000 7f69c91ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.245+0000 7f69c91ec640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T14:05:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.245+0000 7f69c91ec640 1 -- 172.21.15.40:0/3173878169 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40a9490 con 0x7f69c40aa150 2023-07-29T14:05:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.247+0000 7f69c91ec640 1 --2- 172.21.15.40:0/3173878169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69c40aa150 0x7f69c4144cf0 secure :-1 s=READY pgs=3205 cs=0 l=1 rev1=1 crypto rx=0x7f69b40097a0 tx=0x7f69b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea7e97bc90452f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:45.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 1 -- 172.21.15.40:0/3173878169 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4031830 con 0x7f69c40aa150 2023-07-29T14:05:45.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:45.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T14:05:45.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 10 monclient: dump: 2023-07-29T14:05:45.255 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:45.255 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:45.255 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.255 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.256 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:45.256 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.256 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:45.256 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:45.256 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:45.256 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 10 monclient: _finish_auth 0 2023-07-29T14:05:45.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 10 monclient: _check_auth_tickets 2023-07-29T14:05:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 1 -- 172.21.15.40:0/3173878169 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b4031990 con 0x7f69c40aa150 2023-07-29T14:05:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 1 -- 172.21.15.40:0/3173878169 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4031d20 con 0x7f69c40aa150 2023-07-29T14:05:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69c37fe640 10 monclient: dump: 2023-07-29T14:05:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:45.258 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:45.258 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.258 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.258 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:45.258 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.258 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:45.258 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:45.259 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:45.259 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69cac4f640 5 monclient: authenticate success, global_id 14883 2023-07-29T14:05:45.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69cac4f640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:45.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.252+0000 7f69cac4f640 4 set_mon_vals no callback set 2023-07-29T14:05:45.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 10 monclient: shutdown 2023-07-29T14:05:45.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:45.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 -- 172.21.15.40:0/3173878169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69c40aa150 msgr2=0x7f69c4144cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:45.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 --2- 172.21.15.40:0/3173878169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69c40aa150 0x7f69c4144cf0 secure :-1 s=READY pgs=3205 cs=0 l=1 rev1=1 crypto rx=0x7f69b40097a0 tx=0x7f69b402b8d0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 -- 172.21.15.40:0/3173878169 shutdown_connections 2023-07-29T14:05:45.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 --2- 172.21.15.40:0/3173878169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69c40aa150 0x7f69c4144cf0 unknown :-1 s=CLOSED pgs=3205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 -- 172.21.15.40:0/3173878169 >> 172.21.15.40:0/3173878169 conn(0x7f69c4065b00 msgr2=0x7f69c4065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:45.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 -- 172.21.15.40:0/3173878169 shutdown_connections 2023-07-29T14:05:45.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 -- 172.21.15.40:0/3173878169 wait complete. 2023-07-29T14:05:45.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 10 monclient: build_initial_monmap 2023-07-29T14:05:45.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 10 monclient: monmap: 2023-07-29T14:05:45.261 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:45.261 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:45.261 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:45.261 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:45.262 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:45.262 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.262 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:45.262 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:45.262 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:45.262 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 librados: starting msgr at 2023-07-29T14:05:45.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 librados: starting objecter 2023-07-29T14:05:45.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 Processor -- start 2023-07-29T14:05:45.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 -- start start 2023-07-29T14:05:45.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 librados: setting wanted keys 2023-07-29T14:05:45.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 1 librados: calling monclient init 2023-07-29T14:05:45.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.254+0000 7f69cac4f640 10 monclient: init 2023-07-29T14:05:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T14:05:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c40aa150 0x7f69c4143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 10 monclient(hunting): picked mon.noname-b con 0x7f69c40aa150 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f69c40b74a0 0x7f69c40b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 10 monclient(hunting): picked mon.noname-c con 0x7f69c40b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69c413e4f0 0x7f69c41d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:45.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 10 monclient(hunting): picked mon.noname-a con 0x7f69c413e4f0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:45.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:45.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40ab240 con 0x7f69c413e4f0 2023-07-29T14:05:45.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:45.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c4143f30 con 0x7f69c40aa150 2023-07-29T14:05:45.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:45.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40aad20 con 0x7f69c40b74a0 2023-07-29T14:05:45.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:45.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69cac4f640 10 monclient(hunting): authenticate will time out at 3667.388196s 2023-07-29T14:05:45.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c91ec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69c413e4f0 0x7f69c41d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:45.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f69c40b74a0 0x7f69c40b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:45.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f69c40b74a0 0x7f69c40b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49428/0 (socket says 172.21.15.40:49428) 2023-07-29T14:05:45.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c3fff640 1 -- 172.21.15.40:0/3114247605 learned_addr learned my addr 172.21.15.40:0/3114247605 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:45.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c3fff640 10 monclient(hunting): get_auth_request con 0x7f69c40b74a0 auth_method 0 2023-07-29T14:05:45.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:45.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c3fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:45.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c3fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:45.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c91ec640 10 monclient(hunting): get_auth_request con 0x7f69c413e4f0 auth_method 0 2023-07-29T14:05:45.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c91ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:45.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c91ec640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:45.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c91ec640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:45.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c89eb640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c40aa150 0x7f69c4143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:45.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c89eb640 10 monclient(hunting): get_auth_request con 0x7f69c40aa150 auth_method 0 2023-07-29T14:05:45.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c89eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:45.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c89eb640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:45.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c89eb640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:45.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c91ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:45.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c91ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:45.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c91ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:45.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:45.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:45.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.256+0000 7f69c3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:45.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.258+0000 7f69c89eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:45.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.258+0000 7f69c89eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:45.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.258+0000 7f69c89eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:45.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.260+0000 7f69c91ec640 10 monclient(hunting): handle_auth_done global_id 14889 payload 824 2023-07-29T14:05:45.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.260+0000 7f69c91ec640 1 -- 172.21.15.40:0/3114247605 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f69c40b74a0 msgr2=0x7f69c40b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:45.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.260+0000 7f69c91ec640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f69c40b74a0 0x7f69c40b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.260+0000 7f69c91ec640 1 -- 172.21.15.40:0/3114247605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c40aa150 msgr2=0x7f69c4143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:45.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.260+0000 7f69c91ec640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c40aa150 0x7f69c4143400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.260+0000 7f69c91ec640 10 monclient: _finish_hunting 0 2023-07-29T14:05:45.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.260+0000 7f69c91ec640 1 monclient: found mon.noname-a 2023-07-29T14:05:45.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.260+0000 7f69c91ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:45.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.260+0000 7f69c91ec640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T14:05:45.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.260+0000 7f69c91ec640 1 -- 172.21.15.40:0/3114247605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b4009400 con 0x7f69c413e4f0 2023-07-29T14:05:45.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.262+0000 7f69c91ec640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69c413e4f0 0x7f69c41d5a80 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 crypto rx=0x7f69b4033000 tx=0x7f69b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:45.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.262+0000 7f69c17fa640 10 client.?.objecter ms_handle_connect 0x7f69c413e4f0 2023-07-29T14:05:45.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.262+0000 7f69c17fa640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:45.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 1 -- 172.21.15.40:0/3114247605 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b403a6f0 con 0x7f69c413e4f0 2023-07-29T14:05:45.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:45.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T14:05:45.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 10 monclient: dump: 2023-07-29T14:05:45.288 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:45.288 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:45.288 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.289 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.289 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:45.289 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.289 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:45.289 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:45.289 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:45.289 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 10 monclient: _finish_auth 0 2023-07-29T14:05:45.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 10 monclient: _check_auth_tickets 2023-07-29T14:05:45.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:45.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 1 -- 172.21.15.40:0/3114247605 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b403a890 con 0x7f69c413e4f0 2023-07-29T14:05:45.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:45.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 1 -- 172.21.15.40:0/3114247605 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b403ac40 con 0x7f69c413e4f0 2023-07-29T14:05:45.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:45.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c17fa640 10 monclient: dump: 2023-07-29T14:05:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:45.292 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69cac4f640 5 monclient: authenticate success, global_id 14889 2023-07-29T14:05:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69cac4f640 10 monclient: _renew_subs 2023-07-29T14:05:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69cac4f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69cac4f640 1 -- 172.21.15.40:0/3114247605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c40aa8d0 con 0x7f69c413e4f0 2023-07-29T14:05:45.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69c2ffd640 4 set_mon_vals no callback set 2023-07-29T14:05:45.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69cac4f640 10 client.14889.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:45.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69cac4f640 10 monclient: _renew_subs 2023-07-29T14:05:45.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69cac4f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:45.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69cac4f640 1 -- 172.21.15.40:0/3114247605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c41d61a0 con 0x7f69c413e4f0 2023-07-29T14:05:45.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.265+0000 7f69cac4f640 1 librados: init done 2023-07-29T14:05:45.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.266+0000 7f69cac4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:45.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.266+0000 7f69cac4f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:45.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.266+0000 7f69cac4f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:45.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.266+0000 7f69cac4f640 1 -- 172.21.15.40:0/3114247605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c40aaf40 con 0x7f69c413e4f0 2023-07-29T14:05:45.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.285+0000 7f69c17fa640 1 -- 172.21.15.40:0/3114247605 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f69b4043410 con 0x7f69c413e4f0 2023-07-29T14:05:45.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.286+0000 7f69c17fa640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f69b006bfb0 0x7f69b006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:45.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.286+0000 7f69c89eb640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f69b006bfb0 0x7f69b006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:45.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.286+0000 7f69c89eb640 10 monclient: get_auth_request con 0x7f69b006bfb0 auth_method 0 2023-07-29T14:05:45.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.287+0000 7f69c89eb640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f69b006bfb0 0x7f69b006e460 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0015b0 tx=0x7f69bc009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:45.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.287+0000 7f69c17fa640 10 client.14889.objecter ms_handle_connect 0x7f69b006bfb0 2023-07-29T14:05:45.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.288+0000 7f69c17fa640 1 -- 172.21.15.40:0/3114247605 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f69b407dc20 con 0x7f69c413e4f0 2023-07-29T14:05:45.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.288+0000 7f69c17fa640 10 client.14889.objecter ms_dispatch 0x7f69c4000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:45.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.288+0000 7f69c17fa640 3 client.14889.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:45.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.288+0000 7f69c17fa640 3 client.14889.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:45.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.288+0000 7f69c17fa640 20 client.14889.objecter dump_active .. 0 homeless 2023-07-29T14:05:45.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.536+0000 7f69c17fa640 1 -- 172.21.15.40:0/3114247605 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f69b4076050 con 0x7f69c413e4f0 2023-07-29T14:05:45.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.536+0000 7f69c17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:45.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.536+0000 7f69c17fa640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:45.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.716+0000 7f69cac4f640 1 -- 172.21.15.40:0/3114247605 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69c40aa750 con 0x7f69b006bfb0 2023-07-29T14:05:45.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.721+0000 7f69c17fa640 1 -- 172.21.15.40:0/3114247605 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152049 (secure 0 0 0) 0x7f69bc00fe00 con 0x7f69b006bfb0 2023-07-29T14:05:45.732 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T14:05:45.733 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-29T14:05:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 10 librados: watch_flush enter 2023-07-29T14:05:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 10 librados: watch_flush exit 2023-07-29T14:05:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 20 client.14889.objecter shutdown clearing up homeless session... 2023-07-29T14:05:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 10 client.14889.objecter successfully canceled tick 2023-07-29T14:05:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 -- 172.21.15.40:0/3114247605 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f69b006bfb0 msgr2=0x7f69b006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f69b006bfb0 0x7f69b006e460 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0015b0 tx=0x7f69bc009040 comp rx=0 tx=0).stop 2023-07-29T14:05:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 10 monclient: shutdown 2023-07-29T14:05:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 -- 172.21.15.40:0/3114247605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69c413e4f0 msgr2=0x7f69c41d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69c413e4f0 0x7f69c41d5a80 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 crypto rx=0x7f69b4033000 tx=0x7f69b40046c0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 -- 172.21.15.40:0/3114247605 shutdown_connections 2023-07-29T14:05:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f69b006bfb0 0x7f69b006e460 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69c413e4f0 0x7f69c41d5a80 unknown :-1 s=CLOSED pgs=3206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f69c40b74a0 0x7f69c40b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 --2- 172.21.15.40:0/3114247605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c40aa150 0x7f69c4143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 -- 172.21.15.40:0/3114247605 >> 172.21.15.40:0/3114247605 conn(0x7f69c4065b00 msgr2=0x7f69c413b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:45.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 -- 172.21.15.40:0/3114247605 shutdown_connections 2023-07-29T14:05:45.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 -- 172.21.15.40:0/3114247605 wait complete. 2023-07-29T14:05:45.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.725+0000 7f69cac4f640 1 librados: shutdown 2023-07-29T14:05:45.739 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":2134,"stamp":"2023-07-29T14:05:45.630383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":523924,"kb_used_data":50744,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754450796,"statfs":{"total":773094113280,"available":772557615104,"internally_reserved":0,"allocated":51961856,"data_stored":28370954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":132,"apply_latency_ms":132,"commit_latency_ns":132000000,"apply_latency_ns":132000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.154570"},"pg_stats":[{"pgid":"7.1b","version":"0'0","reported_seq":969,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.563113+0000","last_change":"2023-07-29T13:56:55.662556+0000","last_active":"2023-07-29T14:05:37.563113+0000","last_peered":"2023-07-29T14:05:37.563113+0000","last_clean":"2023-07-29T14:05:37.563113+0000","last_became_active":"2023-07-29T13:56:55.656930+0000","last_became_peered":"2023-07-29T13:56:55.656930+0000","last_unstale":"2023-07-29T14:05:37.563113+0000","last_undegraded":"2023-07-29T14:05:37.563113+0000","last_fullsized":"2023-07-29T14:05:37.563113+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:52:51.321547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":978,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.410164+0000","last_change":"2023-07-29T13:41:38.328763+0000","last_active":"2023-07-29T14:05:37.410164+0000","last_peered":"2023-07-29T14:05:37.410164+0000","last_clean":"2023-07-29T14:05:37.410164+0000","last_became_active":"2023-07-29T13:41:38.322679+0000","last_became_peered":"2023-07-29T13:41:38.322679+0000","last_unstale":"2023-07-29T14:05:37.410164+0000","last_undegraded":"2023-07-29T14:05:37.410164+0000","last_fullsized":"2023-07-29T14:05:37.410164+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:40:55.886733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":965,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.400523+0000","last_change":"2023-07-29T13:29:52.105383+0000","last_active":"2023-07-29T14:05:37.400523+0000","last_peered":"2023-07-29T14:05:37.400523+0000","last_clean":"2023-07-29T14:05:37.400523+0000","last_became_active":"2023-07-29T13:29:52.099163+0000","last_became_peered":"2023-07-29T13:29:52.099163+0000","last_unstale":"2023-07-29T14:05:37.400523+0000","last_undegraded":"2023-07-29T14:05:37.400523+0000","last_fullsized":"2023-07-29T14:05:37.400523+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:03:36.535099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":932,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.793552+0000","last_change":"2023-07-29T13:29:41.816664+0000","last_active":"2023-07-29T14:05:39.793552+0000","last_peered":"2023-07-29T14:05:39.793552+0000","last_clean":"2023-07-29T14:05:39.793552+0000","last_became_active":"2023-07-29T13:29:41.782506+0000","last_became_peered":"2023-07-29T13:29:41.782506+0000","last_unstale":"2023-07-29T14:05:39.793552+0000","last_undegraded":"2023-07-29T14:05:39.793552+0000","last_fullsized":"2023-07-29T14:05:39.793552+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:20:57.566653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":948,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.074570+0000","last_change":"2023-07-29T14:05:38.066403+0000","last_active":"2023-07-29T14:05:38.074570+0000","last_peered":"2023-07-29T14:05:38.074570+0000","last_clean":"2023-07-29T14:05:38.074570+0000","last_became_active":"2023-07-29T14:05:38.060277+0000","last_became_peered":"2023-07-29T14:05:38.060277+0000","last_unstale":"2023-07-29T14:05:38.074570+0000","last_undegraded":"2023-07-29T14:05:38.074570+0000","last_fullsized":"2023-07-29T14:05:38.074570+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:50:25.119138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":914,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.478515+0000","last_change":"2023-07-29T13:53:19.606247+0000","last_active":"2023-07-29T14:05:37.478515+0000","last_peered":"2023-07-29T14:05:37.478515+0000","last_clean":"2023-07-29T14:05:37.478515+0000","last_became_active":"2023-07-29T13:53:19.600386+0000","last_became_peered":"2023-07-29T13:53:19.600386+0000","last_unstale":"2023-07-29T14:05:37.478515+0000","last_undegraded":"2023-07-29T14:05:37.478515+0000","last_fullsized":"2023-07-29T14:05:37.478515+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:46:03.055229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":937,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.571721+0000","last_change":"2023-07-29T13:56:55.406251+0000","last_active":"2023-07-29T14:05:37.571721+0000","last_peered":"2023-07-29T14:05:37.571721+0000","last_clean":"2023-07-29T14:05:37.571721+0000","last_became_active":"2023-07-29T13:56:55.400591+0000","last_became_peered":"2023-07-29T13:56:55.400591+0000","last_unstale":"2023-07-29T14:05:37.571721+0000","last_undegraded":"2023-07-29T14:05:37.571721+0000","last_fullsized":"2023-07-29T14:05:37.571721+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:40:48.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":955,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.503487+0000","last_change":"2023-07-29T13:29:31.089676+0000","last_active":"2023-07-29T14:05:37.503487+0000","last_peered":"2023-07-29T14:05:37.503487+0000","last_clean":"2023-07-29T14:05:37.503487+0000","last_became_active":"2023-07-29T13:29:31.072042+0000","last_became_peered":"2023-07-29T13:29:31.072042+0000","last_unstale":"2023-07-29T14:05:37.503487+0000","last_undegraded":"2023-07-29T14:05:37.503487+0000","last_fullsized":"2023-07-29T14:05:37.503487+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:28:28.141003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":913,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.480494+0000","last_change":"2023-07-29T13:53:19.677067+0000","last_active":"2023-07-29T14:05:37.480494+0000","last_peered":"2023-07-29T14:05:37.480494+0000","last_clean":"2023-07-29T14:05:37.480494+0000","last_became_active":"2023-07-29T13:53:19.671215+0000","last_became_peered":"2023-07-29T13:53:19.671215+0000","last_unstale":"2023-07-29T14:05:37.480494+0000","last_undegraded":"2023-07-29T14:05:37.480494+0000","last_fullsized":"2023-07-29T14:05:37.480494+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:51:18.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":987,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.432372+0000","last_change":"2023-07-29T13:56:55.517147+0000","last_active":"2023-07-29T14:05:37.432372+0000","last_peered":"2023-07-29T14:05:37.432372+0000","last_clean":"2023-07-29T14:05:37.432372+0000","last_became_active":"2023-07-29T13:56:55.511490+0000","last_became_peered":"2023-07-29T13:56:55.511490+0000","last_unstale":"2023-07-29T14:05:37.432372+0000","last_undegraded":"2023-07-29T14:05:37.432372+0000","last_fullsized":"2023-07-29T14:05:37.432372+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:26.433868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":969,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.634817+0000","last_change":"2023-07-29T14:05:37.627583+0000","last_active":"2023-07-29T14:05:37.634817+0000","last_peered":"2023-07-29T14:05:37.634817+0000","last_clean":"2023-07-29T14:05:37.634817+0000","last_became_active":"2023-07-29T14:05:37.617822+0000","last_became_peered":"2023-07-29T14:05:37.617822+0000","last_unstale":"2023-07-29T14:05:37.634817+0000","last_undegraded":"2023-07-29T14:05:37.634817+0000","last_fullsized":"2023-07-29T14:05:37.634817+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:03:53.476296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":914,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.555235+0000","last_change":"2023-07-29T13:53:19.644608+0000","last_active":"2023-07-29T14:05:39.555235+0000","last_peered":"2023-07-29T14:05:39.555235+0000","last_clean":"2023-07-29T14:05:39.555235+0000","last_became_active":"2023-07-29T13:53:19.637473+0000","last_became_peered":"2023-07-29T13:53:19.637473+0000","last_unstale":"2023-07-29T14:05:39.555235+0000","last_undegraded":"2023-07-29T14:05:39.555235+0000","last_fullsized":"2023-07-29T14:05:39.555235+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:53:02.384899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":933,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.555513+0000","last_change":"2023-07-29T13:29:33.530600+0000","last_active":"2023-07-29T14:05:37.555513+0000","last_peered":"2023-07-29T14:05:37.555513+0000","last_clean":"2023-07-29T14:05:37.555513+0000","last_became_active":"2023-07-29T13:29:33.515255+0000","last_became_peered":"2023-07-29T13:29:33.515255+0000","last_unstale":"2023-07-29T14:05:37.555513+0000","last_undegraded":"2023-07-29T14:05:37.555513+0000","last_fullsized":"2023-07-29T14:05:37.555513+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:21:56.794070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":986,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.524744+0000","last_change":"2023-07-29T14:05:37.517334+0000","last_active":"2023-07-29T14:05:37.524744+0000","last_peered":"2023-07-29T14:05:37.524744+0000","last_clean":"2023-07-29T14:05:37.524744+0000","last_became_active":"2023-07-29T14:05:37.510437+0000","last_became_peered":"2023-07-29T14:05:37.510437+0000","last_unstale":"2023-07-29T14:05:37.524744+0000","last_undegraded":"2023-07-29T14:05:37.524744+0000","last_fullsized":"2023-07-29T14:05:37.524744+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:38:04.875711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":939,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.573766+0000","last_change":"2023-07-29T13:56:55.146603+0000","last_active":"2023-07-29T14:05:37.573766+0000","last_peered":"2023-07-29T14:05:37.573766+0000","last_clean":"2023-07-29T14:05:37.573766+0000","last_became_active":"2023-07-29T13:56:55.139167+0000","last_became_peered":"2023-07-29T13:56:55.139167+0000","last_unstale":"2023-07-29T14:05:37.573766+0000","last_undegraded":"2023-07-29T14:05:37.573766+0000","last_fullsized":"2023-07-29T14:05:37.573766+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:56:23.477235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":951,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.627838+0000","last_change":"2023-07-29T14:05:37.621178+0000","last_active":"2023-07-29T14:05:37.627838+0000","last_peered":"2023-07-29T14:05:37.627838+0000","last_clean":"2023-07-29T14:05:37.627838+0000","last_became_active":"2023-07-29T14:05:37.614802+0000","last_became_peered":"2023-07-29T14:05:37.614802+0000","last_unstale":"2023-07-29T14:05:37.627838+0000","last_undegraded":"2023-07-29T14:05:37.627838+0000","last_fullsized":"2023-07-29T14:05:37.627838+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:42:16.790121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":990,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.389647+0000","last_change":"2023-07-29T13:56:55.073408+0000","last_active":"2023-07-29T14:05:38.389647+0000","last_peered":"2023-07-29T14:05:38.389647+0000","last_clean":"2023-07-29T14:05:38.389647+0000","last_became_active":"2023-07-29T13:56:55.065724+0000","last_became_peered":"2023-07-29T13:56:55.065724+0000","last_unstale":"2023-07-29T14:05:38.389647+0000","last_undegraded":"2023-07-29T14:05:38.389647+0000","last_fullsized":"2023-07-29T14:05:38.389647+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:42:58.825948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":972,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.422249+0000","last_change":"2023-07-29T13:28:14.434312+0000","last_active":"2023-07-29T14:05:37.422249+0000","last_peered":"2023-07-29T14:05:37.422249+0000","last_clean":"2023-07-29T14:05:37.422249+0000","last_became_active":"2023-07-29T13:28:14.423044+0000","last_became_peered":"2023-07-29T13:28:14.423044+0000","last_unstale":"2023-07-29T14:05:37.422249+0000","last_undegraded":"2023-07-29T14:05:37.422249+0000","last_fullsized":"2023-07-29T14:05:37.422249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:56:27.865715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1","version":"70'7","reported_seq":984,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.521327+0000","last_change":"2023-07-29T13:41:38.528285+0000","last_active":"2023-07-29T14:05:37.521327+0000","last_peered":"2023-07-29T14:05:37.521327+0000","last_clean":"2023-07-29T14:05:37.521327+0000","last_became_active":"2023-07-29T13:41:38.521847+0000","last_became_peered":"2023-07-29T13:41:38.521847+0000","last_unstale":"2023-07-29T14:05:37.521327+0000","last_undegraded":"2023-07-29T14:05:37.521327+0000","last_fullsized":"2023-07-29T14:05:37.521327+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:08:42.836378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":945,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.566657+0000","last_change":"2023-07-29T13:56:55.263959+0000","last_active":"2023-07-29T14:05:37.566657+0000","last_peered":"2023-07-29T14:05:37.566657+0000","last_clean":"2023-07-29T14:05:37.566657+0000","last_became_active":"2023-07-29T13:56:55.258241+0000","last_became_peered":"2023-07-29T13:56:55.258241+0000","last_unstale":"2023-07-29T14:05:37.566657+0000","last_undegraded":"2023-07-29T14:05:37.566657+0000","last_fullsized":"2023-07-29T14:05:37.566657+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:42.517469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":967,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.437586+0000","last_change":"2023-07-29T13:29:52.230014+0000","last_active":"2023-07-29T14:05:37.437586+0000","last_peered":"2023-07-29T14:05:37.437586+0000","last_clean":"2023-07-29T14:05:37.437586+0000","last_became_active":"2023-07-29T13:29:52.224467+0000","last_became_peered":"2023-07-29T13:29:52.224467+0000","last_unstale":"2023-07-29T14:05:37.437586+0000","last_undegraded":"2023-07-29T14:05:37.437586+0000","last_fullsized":"2023-07-29T14:05:37.437586+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:51:03.019114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":974,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.636002+0000","last_change":"2023-07-29T13:28:14.652586+0000","last_active":"2023-07-29T14:05:39.636002+0000","last_peered":"2023-07-29T14:05:39.636002+0000","last_clean":"2023-07-29T14:05:39.636002+0000","last_became_active":"2023-07-29T13:28:14.642023+0000","last_became_peered":"2023-07-29T13:28:14.642023+0000","last_unstale":"2023-07-29T14:05:39.636002+0000","last_undegraded":"2023-07-29T14:05:39.636002+0000","last_fullsized":"2023-07-29T14:05:39.636002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:25:23.334956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":934,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.566809+0000","last_change":"2023-07-29T13:29:41.869711+0000","last_active":"2023-07-29T14:05:37.566809+0000","last_peered":"2023-07-29T14:05:37.566809+0000","last_clean":"2023-07-29T14:05:37.566809+0000","last_became_active":"2023-07-29T13:29:41.844314+0000","last_became_peered":"2023-07-29T13:29:41.844314+0000","last_unstale":"2023-07-29T14:05:37.566809+0000","last_undegraded":"2023-07-29T14:05:37.566809+0000","last_fullsized":"2023-07-29T14:05:37.566809+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:41:40.503015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":964,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.476599+0000","last_change":"2023-07-29T13:56:55.159956+0000","last_active":"2023-07-29T14:05:37.476599+0000","last_peered":"2023-07-29T14:05:37.476599+0000","last_clean":"2023-07-29T14:05:37.476599+0000","last_became_active":"2023-07-29T13:56:55.150514+0000","last_became_peered":"2023-07-29T13:56:55.150514+0000","last_unstale":"2023-07-29T14:05:37.476599+0000","last_undegraded":"2023-07-29T14:05:37.476599+0000","last_fullsized":"2023-07-29T14:05:37.476599+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:16:38.329000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1006,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.420320+0000","last_change":"2023-07-29T13:56:55.615681+0000","last_active":"2023-07-29T14:05:37.420320+0000","last_peered":"2023-07-29T14:05:37.420320+0000","last_clean":"2023-07-29T14:05:37.420320+0000","last_became_active":"2023-07-29T13:56:55.609414+0000","last_became_peered":"2023-07-29T13:56:55.609414+0000","last_unstale":"2023-07-29T14:05:37.420320+0000","last_undegraded":"2023-07-29T14:05:37.420320+0000","last_fullsized":"2023-07-29T14:05:37.420320+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:36:30.601005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1013,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.391985+0000","last_change":"2023-07-29T13:56:55.550112+0000","last_active":"2023-07-29T14:05:37.391985+0000","last_peered":"2023-07-29T14:05:37.391985+0000","last_clean":"2023-07-29T14:05:37.391985+0000","last_became_active":"2023-07-29T13:56:55.543861+0000","last_became_peered":"2023-07-29T13:56:55.543861+0000","last_unstale":"2023-07-29T14:05:37.391985+0000","last_undegraded":"2023-07-29T14:05:37.391985+0000","last_fullsized":"2023-07-29T14:05:37.391985+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:57:36.179121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":951,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.620825+0000","last_change":"2023-07-29T14:05:37.613527+0000","last_active":"2023-07-29T14:05:37.620825+0000","last_peered":"2023-07-29T14:05:37.620825+0000","last_clean":"2023-07-29T14:05:37.620825+0000","last_became_active":"2023-07-29T14:05:37.607023+0000","last_became_peered":"2023-07-29T14:05:37.607023+0000","last_unstale":"2023-07-29T14:05:37.620825+0000","last_undegraded":"2023-07-29T14:05:37.620825+0000","last_fullsized":"2023-07-29T14:05:37.620825+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:07:39.130065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":974,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.659241+0000","last_change":"2023-07-29T13:28:14.713485+0000","last_active":"2023-07-29T14:05:39.659241+0000","last_peered":"2023-07-29T14:05:39.659241+0000","last_clean":"2023-07-29T14:05:39.659241+0000","last_became_active":"2023-07-29T13:28:14.708042+0000","last_became_peered":"2023-07-29T13:28:14.708042+0000","last_unstale":"2023-07-29T14:05:39.659241+0000","last_undegraded":"2023-07-29T14:05:39.659241+0000","last_fullsized":"2023-07-29T14:05:39.659241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:25:56.525512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":1102,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.391897+0000","last_change":"2023-07-29T13:56:55.164457+0000","last_active":"2023-07-29T14:05:38.391897+0000","last_peered":"2023-07-29T14:05:38.391897+0000","last_clean":"2023-07-29T14:05:38.391897+0000","last_became_active":"2023-07-29T13:56:55.157702+0000","last_became_peered":"2023-07-29T13:56:55.157702+0000","last_unstale":"2023-07-29T14:05:38.391897+0000","last_undegraded":"2023-07-29T14:05:38.391897+0000","last_fullsized":"2023-07-29T14:05:38.391897+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_clean_scrub_stamp":"2023-07-29T13:28:06.434639+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:06:43.352094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":996,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.500941+0000","last_change":"2023-07-29T13:56:55.347022+0000","last_active":"2023-07-29T14:05:37.500941+0000","last_peered":"2023-07-29T14:05:37.500941+0000","last_clean":"2023-07-29T14:05:37.500941+0000","last_became_active":"2023-07-29T13:56:55.341426+0000","last_became_peered":"2023-07-29T13:56:55.341426+0000","last_unstale":"2023-07-29T14:05:37.500941+0000","last_undegraded":"2023-07-29T14:05:37.500941+0000","last_fullsized":"2023-07-29T14:05:37.500941+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:46:20.874037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":947,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.437114+0000","last_change":"2023-07-29T13:41:38.063359+0000","last_active":"2023-07-29T14:05:37.437114+0000","last_peered":"2023-07-29T14:05:37.437114+0000","last_clean":"2023-07-29T14:05:37.437114+0000","last_became_active":"2023-07-29T13:41:38.057649+0000","last_became_peered":"2023-07-29T13:41:38.057649+0000","last_unstale":"2023-07-29T14:05:37.437114+0000","last_undegraded":"2023-07-29T14:05:37.437114+0000","last_fullsized":"2023-07-29T14:05:37.437114+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:36:46.868780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":926,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.484786+0000","last_change":"2023-07-29T13:53:19.566408+0000","last_active":"2023-07-29T14:05:37.484786+0000","last_peered":"2023-07-29T14:05:37.484786+0000","last_clean":"2023-07-29T14:05:37.484786+0000","last_became_active":"2023-07-29T13:53:19.560480+0000","last_became_peered":"2023-07-29T13:53:19.560480+0000","last_unstale":"2023-07-29T14:05:37.484786+0000","last_undegraded":"2023-07-29T14:05:37.484786+0000","last_fullsized":"2023-07-29T14:05:37.484786+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:01:49.170231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":914,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.488215+0000","last_change":"2023-07-29T13:53:19.617225+0000","last_active":"2023-07-29T14:05:37.488215+0000","last_peered":"2023-07-29T14:05:37.488215+0000","last_clean":"2023-07-29T14:05:37.488215+0000","last_became_active":"2023-07-29T13:53:19.611477+0000","last_became_peered":"2023-07-29T13:53:19.611477+0000","last_unstale":"2023-07-29T14:05:37.488215+0000","last_undegraded":"2023-07-29T14:05:37.488215+0000","last_fullsized":"2023-07-29T14:05:37.488215+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:52:56.727655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":951,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.539929+0000","last_change":"2023-07-29T13:56:55.182600+0000","last_active":"2023-07-29T14:05:37.539929+0000","last_peered":"2023-07-29T14:05:37.539929+0000","last_clean":"2023-07-29T14:05:37.539929+0000","last_became_active":"2023-07-29T13:56:55.173228+0000","last_became_peered":"2023-07-29T13:56:55.173228+0000","last_unstale":"2023-07-29T14:05:37.539929+0000","last_undegraded":"2023-07-29T14:05:37.539929+0000","last_fullsized":"2023-07-29T14:05:37.539929+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:57:08.005958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":938,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.548791+0000","last_change":"2023-07-29T13:56:55.243737+0000","last_active":"2023-07-29T14:05:37.548791+0000","last_peered":"2023-07-29T14:05:37.548791+0000","last_clean":"2023-07-29T14:05:37.548791+0000","last_became_active":"2023-07-29T13:56:55.238036+0000","last_became_peered":"2023-07-29T13:56:55.238036+0000","last_unstale":"2023-07-29T14:05:37.548791+0000","last_undegraded":"2023-07-29T14:05:37.548791+0000","last_fullsized":"2023-07-29T14:05:37.548791+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:04:27.494041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":932,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.553875+0000","last_change":"2023-07-29T13:29:33.527250+0000","last_active":"2023-07-29T14:05:37.553875+0000","last_peered":"2023-07-29T14:05:37.553875+0000","last_clean":"2023-07-29T14:05:37.553875+0000","last_became_active":"2023-07-29T13:29:33.508113+0000","last_became_peered":"2023-07-29T13:29:33.508113+0000","last_unstale":"2023-07-29T14:05:37.553875+0000","last_undegraded":"2023-07-29T14:05:37.553875+0000","last_fullsized":"2023-07-29T14:05:37.553875+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:52:25.906655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":941,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.401769+0000","last_change":"2023-07-29T13:29:32.261353+0000","last_active":"2023-07-29T14:05:38.401769+0000","last_peered":"2023-07-29T14:05:38.401769+0000","last_clean":"2023-07-29T14:05:38.401769+0000","last_became_active":"2023-07-29T13:29:32.123788+0000","last_became_peered":"2023-07-29T13:29:32.123788+0000","last_unstale":"2023-07-29T14:05:38.401769+0000","last_undegraded":"2023-07-29T14:05:38.401769+0000","last_fullsized":"2023-07-29T14:05:38.401769+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:25:55.926424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":937,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.564990+0000","last_change":"2023-07-29T13:56:55.597264+0000","last_active":"2023-07-29T14:05:37.564990+0000","last_peered":"2023-07-29T14:05:37.564990+0000","last_clean":"2023-07-29T14:05:37.564990+0000","last_became_active":"2023-07-29T13:56:55.591631+0000","last_became_peered":"2023-07-29T13:56:55.591631+0000","last_unstale":"2023-07-29T14:05:37.564990+0000","last_undegraded":"2023-07-29T14:05:37.564990+0000","last_fullsized":"2023-07-29T14:05:37.564990+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:55:19.929908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1004,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.418634+0000","last_change":"2023-07-29T13:56:55.572081+0000","last_active":"2023-07-29T14:05:37.418634+0000","last_peered":"2023-07-29T14:05:37.418634+0000","last_clean":"2023-07-29T14:05:37.418634+0000","last_became_active":"2023-07-29T13:56:55.564155+0000","last_became_peered":"2023-07-29T13:56:55.564155+0000","last_unstale":"2023-07-29T14:05:37.418634+0000","last_undegraded":"2023-07-29T14:05:37.418634+0000","last_fullsized":"2023-07-29T14:05:37.418634+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:56:05.370398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":973,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.441262+0000","last_change":"2023-07-29T13:53:19.441193+0000","last_active":"2023-07-29T14:05:37.441262+0000","last_peered":"2023-07-29T14:05:37.441262+0000","last_clean":"2023-07-29T14:05:37.441262+0000","last_became_active":"2023-07-29T13:53:19.433950+0000","last_became_peered":"2023-07-29T13:53:19.433950+0000","last_unstale":"2023-07-29T14:05:37.441262+0000","last_undegraded":"2023-07-29T14:05:37.441262+0000","last_fullsized":"2023-07-29T14:05:37.441262+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:38:39.350563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":959,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.530325+0000","last_change":"2023-07-29T14:05:37.522505+0000","last_active":"2023-07-29T14:05:37.530325+0000","last_peered":"2023-07-29T14:05:37.530325+0000","last_clean":"2023-07-29T14:05:37.530325+0000","last_became_active":"2023-07-29T14:05:37.513918+0000","last_became_peered":"2023-07-29T14:05:37.513918+0000","last_unstale":"2023-07-29T14:05:37.530325+0000","last_undegraded":"2023-07-29T14:05:37.530325+0000","last_fullsized":"2023-07-29T14:05:37.530325+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:34:20.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":17,"seq":73014444473,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59672,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312168,"statfs":{"total":96636764160,"available":96575660032,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":15.343999999999999,"5min":16.634,"15min":170.43700000000001},"min":{"1min":6.3490000000000002,"5min":6.3330000000000002,"15min":4.7590000000000003},"max":{"1min":111.66,"5min":172.44399999999999,"15min":12751.485000000001},"last":8.4399999999999995},{"interface":"front","average":{"1min":16.254000000000001,"5min":16.774000000000001,"15min":173.22},"min":{"1min":6.79,"5min":5.4500000000000002,"15min":5.4500000000000002},"max":{"1min":129.28399999999999,"5min":173.48699999999999,"15min":12747.76},"last":9.4369999999999994}]},{"osd":1,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":18.536000000000001,"5min":1543.261,"15min":525.65800000000002},"min":{"1min":7.0010000000000003,"5min":6.5149999999999997,"15min":6.5149999999999997},"max":{"1min":99.557000000000002,"5min":29038.089,"15min":29038.089},"last":12.125},{"interface":"front","average":{"1min":22.518000000000001,"5min":1548.174,"15min":530.21500000000003},"min":{"1min":6.9089999999999998,"5min":6.4749999999999996,"15min":6.2249999999999996},"max":{"1min":102.327,"5min":29039.032999999999,"15min":29039.032999999999},"last":38.445}]},{"osd":2,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":23.297000000000001,"5min":21.439,"15min":20.452000000000002},"min":{"1min":7.992,"5min":7.3200000000000003,"15min":7.3200000000000003},"max":{"1min":104.566,"5min":249.84800000000001,"15min":784.66200000000003},"last":39.115000000000002},{"interface":"front","average":{"1min":21.670000000000002,"5min":21.084,"15min":19.474},"min":{"1min":8.6280000000000001,"5min":8.3919999999999995,"15min":7.8049999999999997},"max":{"1min":99.706000000000003,"5min":202.64500000000001,"15min":368.62200000000001},"last":11.403}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.91,"5min":25.608000000000001,"15min":478.83800000000002},"min":{"1min":7.0910000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"max":{"1min":101.714,"5min":312.73599999999999,"15min":28381.707999999999},"last":12.787000000000001},{"interface":"front","average":{"1min":20.298999999999999,"5min":27.803999999999998,"15min":474.06299999999999},"min":{"1min":9.3219999999999992,"5min":9.3219999999999992,"15min":8.7520000000000007},"max":{"1min":97.727999999999994,"5min":242.63,"15min":28126.012999999999},"last":30.170000000000002}]},{"osd":5,"last update":"Sat Jul 29 14:05:01 2023","interfaces":[{"interface":"back","average":{"1min":17.222999999999999,"5min":20.141999999999999,"15min":32.993000000000002},"min":{"1min":10.452,"5min":8.4540000000000006,"15min":7.3689999999999998},"max":{"1min":79.867000000000004,"5min":85.578999999999994,"15min":488.72300000000001},"last":10.744},{"interface":"front","average":{"1min":14.161,"5min":19.236999999999998,"15min":21.838000000000001},"min":{"1min":8.1379999999999999,"5min":8.1379999999999999,"15min":7.3840000000000003},"max":{"1min":52.622,"5min":106.634,"15min":128.791},"last":30.861000000000001}]},{"osd":6,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":24.904,"5min":20.061,"15min":22.629999999999999},"min":{"1min":9.1470000000000002,"5min":8.0129999999999999,"15min":7.7779999999999996},"max":{"1min":106.33199999999999,"5min":124.819,"15min":787.18499999999995},"last":14.055},{"interface":"front","average":{"1min":25.198,"5min":20.042000000000002,"15min":22.454999999999998},"min":{"1min":9.8010000000000002,"5min":8.5790000000000006,"15min":8.3170000000000002},"max":{"1min":112.328,"5min":112.328,"15min":646.87599999999998},"last":31.472999999999999}]},{"osd":7,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":27.215,"5min":22.292000000000002,"15min":48.816000000000003},"min":{"1min":11.487,"5min":9.2620000000000005,"15min":9.2620000000000005},"max":{"1min":149.43199999999999,"5min":149.43199999999999,"15min":648.54200000000003},"last":32.210999999999999},{"interface":"front","average":{"1min":23.321000000000002,"5min":20.568999999999999,"15min":38.241999999999997},"min":{"1min":10.385,"5min":9.2300000000000004,"15min":8.7309999999999999},"max":{"1min":75.954999999999998,"5min":75.954999999999998,"15min":364.86500000000001},"last":37.286999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444485,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64136,"kb_used_data":6248,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94307704,"statfs":{"total":96636764160,"available":96571088896,"internally_reserved":0,"allocated":6397952,"data_stored":3446913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":13.247999999999999,"15min":130.82599999999999},"min":{"1min":5.3789999999999996,"5min":5.0209999999999999,"15min":4.476},"max":{"1min":29.634,"5min":93.590999999999994,"15min":10009.493},"last":11.063000000000001},{"interface":"front","average":{"1min":13.936999999999999,"5min":13.526,"15min":131.76599999999999},"min":{"1min":7.1299999999999999,"5min":6.2069999999999999,"15min":5.617},"max":{"1min":37.527999999999999,"5min":99.156999999999996,"15min":9995.5869999999995},"last":16.388000000000002}]},{"osd":1,"last update":"Sat Jul 29 14:04:47 2023","interfaces":[{"interface":"back","average":{"1min":11.255000000000001,"5min":1413.5239999999999,"15min":481.31099999999998},"min":{"1min":6.0899999999999999,"5min":6.0899999999999999,"15min":6.0899999999999999},"max":{"1min":17.850999999999999,"5min":29343.018,"15min":29343.018},"last":15.468999999999999},{"interface":"front","average":{"1min":11.234999999999999,"5min":1411.289,"15min":478.77199999999999},"min":{"1min":7.0209999999999999,"5min":6.5880000000000001,"15min":5.8860000000000001},"max":{"1min":18.347000000000001,"5min":29342.263999999999,"15min":29342.263999999999},"last":7.7919999999999998}]},{"osd":3,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":17.623000000000001,"5min":20.777000000000001,"15min":365.04399999999998},"min":{"1min":6.7809999999999997,"5min":6.7809999999999997,"15min":6.7809999999999997},"max":{"1min":86.926000000000002,"5min":331.92399999999998,"15min":29770.313999999998},"last":9.6850000000000005},{"interface":"front","average":{"1min":21.684999999999999,"5min":22.039999999999999,"15min":358.54500000000002},"min":{"1min":9.3070000000000004,"5min":8.0540000000000003,"15min":6.4749999999999996},"max":{"1min":89.156000000000006,"5min":332.49900000000002,"15min":29586.387999999999},"last":17.302}]},{"osd":4,"last update":"Sat Jul 29 14:04:53 2023","interfaces":[{"interface":"back","average":{"1min":12.471,"5min":819.34000000000003,"15min":870.13199999999995},"min":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"max":{"1min":52.584000000000003,"5min":25685.629000000001,"15min":32071.659},"last":11.618},{"interface":"front","average":{"1min":15.125999999999999,"5min":814.09699999999998,"15min":870.976},"min":{"1min":8.8330000000000002,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":35.020000000000003,"5min":25610.863000000001,"15min":32126.150000000001},"last":18.202999999999999}]},{"osd":5,"last update":"Sat Jul 29 14:05:06 2023","interfaces":[{"interface":"back","average":{"1min":14.398,"5min":20.428999999999998,"15min":38.106999999999999},"min":{"1min":8.7579999999999991,"5min":8.4269999999999996,"15min":8.4269999999999996},"max":{"1min":18.667000000000002,"5min":193.45400000000001,"15min":428.04000000000002},"last":10.387},{"interface":"front","average":{"1min":12.019,"5min":20.446999999999999,"15min":41.822000000000003},"min":{"1min":6.891,"5min":6.891,"15min":6.891},"max":{"1min":19.634,"5min":174.292,"15min":430.15499999999997},"last":12.253}]},{"osd":6,"last update":"Sat Jul 29 14:05:40 2023","interfaces":[{"interface":"back","average":{"1min":17.140000000000001,"5min":21.350999999999999,"15min":20.024000000000001},"min":{"1min":8.3450000000000006,"5min":8.3450000000000006,"15min":8.3450000000000006},"max":{"1min":77.855000000000004,"5min":318.45299999999997,"15min":318.45299999999997},"last":12.85},{"interface":"front","average":{"1min":17.119,"5min":21.058,"15min":20.518999999999998},"min":{"1min":9.2230000000000008,"5min":9.2230000000000008,"15min":7.5300000000000002},"max":{"1min":44.755000000000003,"5min":349.34899999999999,"15min":349.34899999999999},"last":20.885999999999999}]},{"osd":7,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":15.279999999999999,"5min":20.536000000000001,"15min":21.027000000000001},"min":{"1min":10.663,"5min":8.5079999999999991,"15min":7.5890000000000004},"max":{"1min":20.757999999999999,"5min":291.80900000000003,"15min":291.80900000000003},"last":13.475},{"interface":"front","average":{"1min":15.644,"5min":21.513000000000002,"15min":22.518000000000001},"min":{"1min":9.6419999999999995,"5min":9.1940000000000008,"15min":9.1940000000000008},"max":{"1min":23.408000000000001,"5min":291.44200000000001,"15min":291.44200000000001},"last":19.187000000000001}]}]},{"osd":1,"up_from":447,"seq":1919850381743,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60432,"kb_used_data":6640,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311408,"statfs":{"total":96636764160,"available":96574881792,"internally_reserved":0,"allocated":6799360,"data_stored":3844753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":20.931000000000001,"5min":20.015999999999998,"15min":112.807},"min":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.3680000000000003},"max":{"1min":116.512,"5min":364.47800000000001,"15min":12525.57},"last":12.721},{"interface":"front","average":{"1min":18.943000000000001,"5min":20.664000000000001,"15min":112.874},"min":{"1min":6.0970000000000004,"5min":6.0970000000000004,"15min":4.6820000000000004},"max":{"1min":100.402,"5min":372.00999999999999,"15min":12523.974},"last":11.795999999999999}]},{"osd":2,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.987,"5min":26.809000000000001,"15min":19.399999999999999},"min":{"1min":7.819,"5min":6.5110000000000001,"15min":5.5919999999999996},"max":{"1min":69.497,"5min":360.88099999999997,"15min":360.88099999999997},"last":13.550000000000001},{"interface":"front","average":{"1min":19.43,"5min":28.155999999999999,"15min":20.445},"min":{"1min":7.093,"5min":7.093,"15min":6.2729999999999997},"max":{"1min":93.792000000000002,"5min":368.51400000000001,"15min":368.51400000000001},"last":17.792999999999999}]},{"osd":3,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":26.696000000000002,"5min":30.559000000000001,"15min":531.69799999999998},"min":{"1min":9.1590000000000007,"5min":6.7140000000000004,"15min":6.7140000000000004},"max":{"1min":216.16300000000001,"5min":435.92200000000003,"15min":34504.942000000003},"last":14.794},{"interface":"front","average":{"1min":31.332000000000001,"5min":41.100999999999999,"15min":539.97299999999996},"min":{"1min":10.423,"5min":10.423,"15min":5.3639999999999999},"max":{"1min":238.33699999999999,"5min":700.10500000000002,"15min":34620.970999999998},"last":21.510999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.427,"5min":1170.991,"15min":980.98599999999999},"min":{"1min":8.4130000000000003,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":50.289000000000001,"5min":25247.098999999998,"15min":32258.529999999999},"last":16.050999999999998},{"interface":"front","average":{"1min":16.536000000000001,"5min":1165.587,"15min":967.39999999999998},"min":{"1min":11.340999999999999,"5min":8.9169999999999998,"15min":6.5090000000000003},"max":{"1min":29.376999999999999,"5min":25174.008999999998,"15min":31933.992999999999},"last":19.478999999999999}]},{"osd":5,"last update":"Sat Jul 29 14:05:13 2023","interfaces":[{"interface":"back","average":{"1min":15.249000000000001,"5min":33.526000000000003,"15min":43.616},"min":{"1min":10.416,"5min":10.294,"15min":10.294},"max":{"1min":23.649999999999999,"5min":433.79300000000001,"15min":433.79300000000001},"last":22.321999999999999},{"interface":"front","average":{"1min":15.952,"5min":36.515000000000001,"15min":46.286000000000001},"min":{"1min":11.467000000000001,"5min":9.6929999999999996,"15min":8.7260000000000009},"max":{"1min":22.300999999999998,"5min":419.22800000000001,"15min":419.22800000000001},"last":18.899999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":16.931999999999999,"5min":21.992000000000001,"15min":19.459},"min":{"1min":6.6079999999999997,"5min":6.327,"15min":6.327},"max":{"1min":154.92400000000001,"5min":357.20499999999998,"15min":357.20499999999998},"last":9.1189999999999998},{"interface":"front","average":{"1min":20.251999999999999,"5min":27.460999999999999,"15min":23.411000000000001},"min":{"1min":10.877000000000001,"5min":9.5079999999999991,"15min":9.5079999999999991},"max":{"1min":155.994,"5min":359.68099999999998,"15min":359.68099999999998},"last":16.553999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.683,"5min":35.137999999999998,"15min":23.725999999999999},"min":{"1min":10.130000000000001,"5min":8.5199999999999996,"15min":7.5919999999999996},"max":{"1min":92.168999999999997,"5min":563.29700000000003,"15min":563.29700000000003},"last":20.231000000000002},{"interface":"front","average":{"1min":17.949000000000002,"5min":30.442,"15min":21.349},"min":{"1min":8.2240000000000002,"5min":8.2240000000000002,"15min":7.2919999999999998},"max":{"1min":113.15900000000001,"5min":352.65199999999999,"15min":352.65199999999999},"last":10.521000000000001}]}]},{"osd":0,"up_from":1002,"seq":4303557231029,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59676,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312164,"statfs":{"total":96636764160,"available":96575655936,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":71.099999999999994,"5min":1023.201,"15min":351.10899999999998},"min":{"1min":8.9689999999999994,"5min":5.9249999999999998,"15min":4.6269999999999998},"max":{"1min":350.00200000000001,"5min":27534,"15min":27534},"last":11.109999999999999},{"interface":"front","average":{"1min":99.875,"5min":1025.7080000000001,"15min":350.98899999999998},"min":{"1min":5.9589999999999996,"5min":5.7240000000000002,"15min":5.0739999999999998},"max":{"1min":591.69600000000003,"5min":27535.008999999998,"15min":27535.008999999998},"last":12.922000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.170999999999999,"5min":22.285,"15min":18.638000000000002},"min":{"1min":11.680999999999999,"5min":8.1069999999999993,"15min":6.6159999999999997},"max":{"1min":364.62799999999999,"5min":364.62799999999999,"15min":364.62799999999999},"last":8.9559999999999995},{"interface":"front","average":{"1min":45.890000000000001,"5min":26.417999999999999,"15min":21.443000000000001},"min":{"1min":14.172000000000001,"5min":10.023999999999999,"15min":8.8780000000000001},"max":{"1min":370.34300000000002,"5min":370.34300000000002,"15min":370.34300000000002},"last":15.214}]},{"osd":3,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":66.832999999999998,"5min":25.178000000000001,"15min":355.89100000000002},"min":{"1min":6.7610000000000001,"5min":6.4409999999999998,"15min":5.891},"max":{"1min":444.59500000000003,"5min":444.59500000000003,"15min":30170.040000000001},"last":12.173},{"interface":"front","average":{"1min":65.754000000000005,"5min":33.606000000000002,"15min":363.07400000000001},"min":{"1min":10.266999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"max":{"1min":386.06099999999998,"5min":386.06099999999998,"15min":30171.194},"last":16.274999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":92.555000000000007,"5min":36.014000000000003,"15min":585.43899999999996},"min":{"1min":12.201000000000001,"5min":6.4480000000000004,"15min":6.0979999999999999},"max":{"1min":445.65300000000002,"5min":445.65300000000002,"15min":29892.669999999998},"last":19.030999999999999},{"interface":"front","average":{"1min":115.806,"5min":40.560000000000002,"15min":588.88900000000001},"min":{"1min":7.923,"5min":5.9400000000000004,"15min":5.9400000000000004},"max":{"1min":593.13499999999999,"5min":593.13499999999999,"15min":29877.289000000001},"last":9.3350000000000009}]},{"osd":5,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.341000000000001,"5min":28.972000000000001,"15min":56.932000000000002},"min":{"1min":12.702,"5min":9.5700000000000003,"15min":9.5700000000000003},"max":{"1min":403.13999999999999,"5min":403.13999999999999,"15min":508.02100000000002},"last":17.283000000000001},{"interface":"front","average":{"1min":43.966999999999999,"5min":23.216000000000001,"15min":32.374000000000002},"min":{"1min":7.6429999999999998,"5min":7.1210000000000004,"15min":7.1210000000000004},"max":{"1min":590.58500000000004,"5min":590.58500000000004,"15min":590.58500000000004},"last":17.960000000000001}]},{"osd":6,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":52.365000000000002,"5min":30.853999999999999,"15min":25.227},"min":{"1min":16.876000000000001,"5min":10.409000000000001,"15min":9.9350000000000005},"max":{"1min":375.36500000000001,"5min":375.36500000000001,"15min":375.36500000000001},"last":11.582000000000001},{"interface":"front","average":{"1min":55.634,"5min":32.579999999999998,"15min":27.417999999999999},"min":{"1min":18.984999999999999,"5min":11.564,"15min":11.564},"max":{"1min":380.31599999999997,"5min":380.31599999999997,"15min":380.31599999999997},"last":19.946000000000002}]},{"osd":7,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":57.960999999999999,"5min":28.638000000000002,"15min":20.780000000000001},"min":{"1min":17.748999999999999,"5min":8.4939999999999998,"15min":7.9349999999999996},"max":{"1min":385.01900000000001,"5min":385.01900000000001,"15min":385.01900000000001},"last":20.901},{"interface":"front","average":{"1min":54.853999999999999,"5min":29.765999999999998,"15min":25.545000000000002},"min":{"1min":10.992000000000001,"5min":8.1229999999999993,"15min":8.1229999999999993},"max":{"1min":446.69099999999997,"5min":446.69099999999997,"15min":446.69099999999997},"last":14.324999999999999}]}]},{"osd":4,"up_from":16,"seq":68719477162,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59676,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312164,"statfs":{"total":96636764160,"available":96575655936,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":13.432,"5min":21.457000000000001,"15min":87.253},"min":{"1min":6.4029999999999996,"5min":4.6230000000000002,"15min":4.6230000000000002},"max":{"1min":48.048999999999999,"5min":495.86900000000003,"15min":10627.831},"last":10.673},{"interface":"front","average":{"1min":18.010000000000002,"5min":25.013000000000002,"15min":90.852000000000004},"min":{"1min":7.2400000000000002,"5min":6.0990000000000002,"15min":5.6660000000000004},"max":{"1min":124.64100000000001,"5min":446.84100000000001,"15min":10653.877},"last":7.1440000000000001}]},{"osd":1,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":11.276999999999999,"5min":1441.567,"15min":495.98200000000003},"min":{"1min":7.2640000000000002,"5min":7.2640000000000002,"15min":5.9909999999999997},"max":{"1min":21.369,"5min":26872.857,"15min":26872.857},"last":12.961},{"interface":"front","average":{"1min":11.013,"5min":1440.0119999999999,"15min":493.40499999999997},"min":{"1min":7.4269999999999996,"5min":7.3090000000000002,"15min":5.6909999999999998},"max":{"1min":17.440999999999999,"5min":26874.048999999999,"15min":26874.048999999999},"last":16.699999999999999}]},{"osd":2,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":12.445,"5min":24.050999999999998,"15min":23.050000000000001},"min":{"1min":8.9000000000000004,"5min":7.3019999999999996,"15min":7.3019999999999996},"max":{"1min":15.904999999999999,"5min":453.40100000000001,"15min":597.11800000000005},"last":13.585000000000001},{"interface":"front","average":{"1min":15.029999999999999,"5min":27.077999999999999,"15min":26.128},"min":{"1min":12.429,"5min":11.631,"15min":8.0559999999999992},"max":{"1min":19.111000000000001,"5min":453.96199999999999,"15min":705.04499999999996},"last":14.6}]},{"osd":3,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":14.789999999999999,"5min":27.626999999999999,"15min":590.51199999999994},"min":{"1min":9.7219999999999995,"5min":7.6799999999999997,"15min":5.9740000000000002},"max":{"1min":23.942,"5min":448.65100000000001,"15min":30725.612000000001},"last":15.250999999999999},{"interface":"front","average":{"1min":12.497,"5min":20.047999999999998,"15min":582.06899999999996},"min":{"1min":7.7690000000000001,"5min":6.3840000000000003,"15min":6.3840000000000003},"max":{"1min":19.815000000000001,"5min":336.62599999999998,"15min":30407.659},"last":17.337}]},{"osd":5,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":16.190999999999999,"5min":30.579000000000001,"15min":32.869},"min":{"1min":14.1,"5min":7.1440000000000001,"15min":7.1440000000000001},"max":{"1min":18.738,"5min":450.81700000000001,"15min":450.81700000000001},"last":15.920999999999999},{"interface":"front","average":{"1min":10.430999999999999,"5min":19.959,"15min":22.803999999999998},"min":{"1min":6.2779999999999996,"5min":6.2779999999999996,"15min":6.2779999999999996},"max":{"1min":20.254999999999999,"5min":340.04000000000002,"15min":340.04000000000002},"last":9.093}]},{"osd":6,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":13.007,"5min":22.312000000000001,"15min":22.866},"min":{"1min":8.1120000000000001,"5min":7.1689999999999996,"15min":6.1520000000000001},"max":{"1min":22.010000000000002,"5min":339.47199999999998,"15min":696.952},"last":9.7759999999999998},{"interface":"front","average":{"1min":14.724,"5min":23.917999999999999,"15min":23.209},"min":{"1min":9.0289999999999999,"5min":6.3449999999999998,"15min":6.3449999999999998},"max":{"1min":20.968,"5min":498.26999999999998,"15min":598.86900000000003},"last":19.806000000000001}]},{"osd":7,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":12.805,"5min":24.097000000000001,"15min":24.802},"min":{"1min":8.7430000000000003,"5min":8.3399999999999999,"15min":7.7930000000000001},"max":{"1min":18.638000000000002,"5min":340.61900000000003,"15min":699.31799999999998},"last":18.100999999999999},{"interface":"front","average":{"1min":17.483000000000001,"5min":31.318999999999999,"15min":33.040999999999997},"min":{"1min":13.861000000000001,"5min":11.067,"15min":11.067},"max":{"1min":19.817,"5min":455.339,"15min":703.68200000000002},"last":11.962}]}]},{"osd":6,"up_from":15,"seq":64424509874,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82180,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94289660,"statfs":{"total":96636764160,"available":96552611840,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":12.891999999999999,"5min":16.984000000000002,"15min":62.423000000000002},"min":{"1min":6.9390000000000001,"5min":5.2350000000000003,"15min":5.2350000000000003},"max":{"1min":29.821000000000002,"5min":224.93899999999999,"15min":9502.5869999999995},"last":5.5629999999999997},{"interface":"front","average":{"1min":15.925000000000001,"5min":19.867000000000001,"15min":65.921999999999997},"min":{"1min":8.2050000000000001,"5min":6.1580000000000004,"15min":6.1580000000000004},"max":{"1min":31.303999999999998,"5min":130.48400000000001,"15min":9511.3719999999994},"last":9.6059999999999999}]},{"osd":1,"last update":"Sat Jul 29 14:04:49 2023","interfaces":[{"interface":"back","average":{"1min":17.140999999999998,"5min":970.20899999999995,"15min":335.46300000000002},"min":{"1min":7.5839999999999996,"5min":7.3369999999999997,"15min":4.8529999999999998},"max":{"1min":73.679000000000002,"5min":26903.921999999999,"15min":26903.921999999999},"last":14.039999999999999},{"interface":"front","average":{"1min":12.878,"5min":963.24699999999996,"15min":332.12599999999998},"min":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.3330000000000002},"max":{"1min":69.432000000000002,"5min":26893.022000000001,"15min":26893.022000000001},"last":11.784000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":14.891999999999999,"5min":21.295999999999999,"15min":19.027000000000001},"min":{"1min":10.173,"5min":7.8609999999999998,"15min":6.6299999999999999},"max":{"1min":26.302,"5min":191.73099999999999,"15min":191.73099999999999},"last":15.919},{"interface":"front","average":{"1min":11.848000000000001,"5min":16.003,"15min":15.422000000000001},"min":{"1min":7.4690000000000003,"5min":5.5499999999999998,"15min":5.5499999999999998},"max":{"1min":18,"5min":189.40199999999999,"15min":189.40199999999999},"last":10.874000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":10.653,"5min":15.525,"15min":521.48800000000006},"min":{"1min":7.2619999999999996,"5min":6.335,"15min":6.335},"max":{"1min":23.292000000000002,"5min":187.852,"15min":30710.619999999999},"last":12.523999999999999},{"interface":"front","average":{"1min":15.202999999999999,"5min":17.899000000000001,"15min":521.02300000000002},"min":{"1min":9.907,"5min":7.7009999999999996,"15min":7.6180000000000003},"max":{"1min":40.335000000000001,"5min":101.88200000000001,"15min":30591.776999999998},"last":17.108000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:04:56 2023","interfaces":[{"interface":"back","average":{"1min":16.030000000000001,"5min":808.61300000000006,"15min":1019.547},"min":{"1min":8.0950000000000006,"5min":6.718,"15min":6.1070000000000002},"max":{"1min":73.055999999999997,"5min":25620.438999999998,"15min":32899.582000000002},"last":8.8569999999999993},{"interface":"front","average":{"1min":16.128,"5min":828.35799999999995,"15min":1009.853},"min":{"1min":10.254,"5min":6.2089999999999996,"15min":6.2089999999999996},"max":{"1min":40.786999999999999,"5min":25979.712,"15min":32485.278999999999},"last":15.188000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:05:04 2023","interfaces":[{"interface":"back","average":{"1min":17.489999999999998,"5min":21.376999999999999,"15min":24.847000000000001},"min":{"1min":11.52,"5min":9.4949999999999992,"15min":8.9290000000000003},"max":{"1min":34.677,"5min":75.25,"15min":189.423},"last":53.837000000000003},{"interface":"front","average":{"1min":14.324,"5min":18.073,"15min":21.690000000000001},"min":{"1min":8.6969999999999992,"5min":7.3330000000000002,"15min":7.3330000000000002},"max":{"1min":24.77,"5min":80.085999999999999,"15min":188.518},"last":52.890000000000001}]},{"osd":7,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":24.321000000000002,"5min":19.489999999999998,"15min":19.814},"min":{"1min":14.164,"5min":7.1059999999999999,"15min":7.1059999999999999},"max":{"1min":74.337000000000003,"5min":74.337000000000003,"15min":200.55099999999999},"last":19.123000000000001},{"interface":"front","average":{"1min":18.581,"5min":20.780000000000001,"15min":22.265000000000001},"min":{"1min":9.6880000000000006,"5min":8.0790000000000006,"15min":8.0790000000000006},"max":{"1min":47.313000000000002,"5min":89.125,"15min":222.65199999999999},"last":17.780999999999999}]}]},{"osd":5,"up_from":704,"seq":3023656976773,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78476,"kb_used_data":6636,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94293364,"statfs":{"total":96636764160,"available":96556404736,"internally_reserved":0,"allocated":6795264,"data_stored":3844748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:04:00 2023","interfaces":[{"interface":"back","average":{"1min":23.48,"5min":22.623000000000001,"15min":41.000999999999998},"min":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":4.742},"max":{"1min":150.63,"5min":262.56099999999998,"15min":299.87799999999999},"last":8.3529999999999998},{"interface":"front","average":{"1min":24.837,"5min":22.727,"15min":52.545999999999999},"min":{"1min":5.8259999999999996,"5min":4.6539999999999999,"15min":4.6539999999999999},"max":{"1min":111.14100000000001,"5min":253.80000000000001,"15min":370.00999999999999},"last":7.3300000000000001}]},{"osd":1,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":13.867000000000001,"5min":25.277000000000001,"15min":33.274999999999999},"min":{"1min":6.5419999999999998,"5min":5.3780000000000001,"15min":5.3780000000000001},"max":{"1min":91.590000000000003,"5min":362.58800000000002,"15min":362.58800000000002},"last":12.525},{"interface":"front","average":{"1min":15.465999999999999,"5min":34.079000000000001,"15min":56.872999999999998},"min":{"1min":7.5650000000000004,"5min":6.8680000000000003,"15min":6.8680000000000003},"max":{"1min":85.760000000000005,"5min":356.69900000000001,"15min":363.50999999999999},"last":9.1530000000000005}]},{"osd":2,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.152000000000001,"5min":26.193000000000001,"15min":51.703000000000003},"min":{"1min":8.1880000000000006,"5min":7.032,"15min":7.032},"max":{"1min":151.804,"5min":312.755,"15min":384.024},"last":18.709},{"interface":"front","average":{"1min":16.754000000000001,"5min":20.221,"15min":25.251000000000001},"min":{"1min":7.782,"5min":7.2759999999999998,"15min":7.2709999999999999},"max":{"1min":102.09099999999999,"5min":228.55600000000001,"15min":228.55600000000001},"last":14.336}]},{"osd":3,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":28.802,"15min":55.957999999999998},"min":{"1min":10.734,"5min":7.4100000000000001,"15min":7.4100000000000001},"max":{"1min":131.36600000000001,"5min":312.12200000000001,"15min":381.23200000000003},"last":14.715},{"interface":"front","average":{"1min":18.568999999999999,"5min":28.875,"15min":66.894999999999996},"min":{"1min":7.944,"5min":7.944,"15min":7.944},"max":{"1min":118.456,"5min":265.92700000000002,"15min":382.779},"last":13.188000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":19.385999999999999,"5min":1108.5630000000001,"15min":412.19600000000003},"min":{"1min":10.5,"5min":7.343,"15min":7.3070000000000004},"max":{"1min":103.768,"5min":23572.741999999998,"15min":23572.741999999998},"last":16.469999999999999},{"interface":"front","average":{"1min":20.039000000000001,"5min":1113.885,"15min":428.73500000000001},"min":{"1min":8.7260000000000009,"5min":6.8929999999999998,"15min":6.8929999999999998},"max":{"1min":137.57400000000001,"5min":23610.142,"15min":23610.142},"last":9.8550000000000004}]},{"osd":6,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":17.074000000000002,"5min":25.704999999999998,"15min":51.972000000000001},"min":{"1min":9.7360000000000007,"5min":8.1709999999999994,"15min":8.1709999999999994},"max":{"1min":41.347000000000001,"5min":263.983,"15min":450.17500000000001},"last":11.878},{"interface":"front","average":{"1min":18.436,"5min":26.684999999999999,"15min":57.575000000000003},"min":{"1min":11.185,"5min":8.1969999999999992,"15min":8.1969999999999992},"max":{"1min":63.707999999999998,"5min":221.99199999999999,"15min":417.303},"last":17.875}]},{"osd":7,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":21.550999999999998,"5min":28.064,"15min":47.530999999999999},"min":{"1min":12.340999999999999,"5min":11.222,"15min":10.602},"max":{"1min":104.718,"5min":284.48000000000002,"15min":341.30500000000001},"last":19.625},{"interface":"front","average":{"1min":18.460999999999999,"5min":27.806000000000001,"15min":53.993000000000002},"min":{"1min":12.089,"5min":11.042999999999999,"15min":11.042999999999999},"max":{"1min":46.353999999999999,"5min":191.33099999999999,"15min":329.86500000000001},"last":15.612}]}]},{"osd":7,"up_from":650,"seq":2791728742801,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59676,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312164,"statfs":{"total":96636764160,"available":96575655936,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:45 2023","interfaces":[{"interface":"back","average":{"1min":11.94,"5min":13.635,"15min":23.015000000000001},"min":{"1min":5.5890000000000004,"5min":5.4539999999999997,"15min":5.4539999999999997},"max":{"1min":28.5,"5min":75.968000000000004,"15min":293.94499999999999},"last":10.49},{"interface":"front","average":{"1min":14.968999999999999,"5min":18.658999999999999,"15min":28.620000000000001},"min":{"1min":6.8179999999999996,"5min":6.8179999999999996,"15min":6.8179999999999996},"max":{"1min":29.143999999999998,"5min":116.952,"15min":358.267},"last":9.1709999999999994}]},{"osd":1,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":19.048999999999999,"5min":1108.855,"15min":389.67099999999999},"min":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":4.6660000000000004},"max":{"1min":184.25800000000001,"5min":27546.963,"15min":27546.963},"last":11.231999999999999},{"interface":"front","average":{"1min":22.969999999999999,"5min":1111.8620000000001,"15min":385.10300000000001},"min":{"1min":8.6110000000000007,"5min":5.5010000000000003,"15min":5.5010000000000003},"max":{"1min":150.75700000000001,"5min":27546.280999999999,"15min":27546.280999999999},"last":9.7629999999999999}]},{"osd":2,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.088000000000001,"5min":19.381,"15min":28.02},"min":{"1min":10.154999999999999,"5min":7.9100000000000001,"15min":7.9100000000000001},"max":{"1min":103.09,"5min":103.09,"15min":461.83800000000002},"last":15.819000000000001},{"interface":"front","average":{"1min":21.068000000000001,"5min":15.704000000000001,"15min":23.861999999999998},"min":{"1min":7.1269999999999998,"5min":6.6239999999999997,"15min":6.6239999999999997},"max":{"1min":181.97900000000001,"5min":181.97900000000001,"15min":385.40800000000002},"last":7.0039999999999996}]},{"osd":3,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":25.065999999999999,"5min":21.029,"15min":1531.4280000000001},"min":{"1min":7.976,"5min":7.6660000000000004,"15min":7.6660000000000004},"max":{"1min":242.197,"5min":242.197,"15min":34550.838000000003},"last":7.8300000000000001},{"interface":"front","average":{"1min":31.428999999999998,"5min":27.873000000000001,"15min":1543.2929999999999},"min":{"1min":12.592000000000001,"5min":6.9550000000000001,"15min":6.9550000000000001},"max":{"1min":240.46899999999999,"5min":243.06399999999999,"15min":34556.381000000001},"last":16.466999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":20.402999999999999,"5min":1074.817,"15min":900.84500000000003},"min":{"1min":6.7220000000000004,"5min":6.3719999999999999,"15min":6.3719999999999999},"max":{"1min":103.931,"5min":25146.41,"15min":27891.68},"last":11.835000000000001},{"interface":"front","average":{"1min":29.045999999999999,"5min":1099.6300000000001,"15min":918.40200000000004},"min":{"1min":9.7360000000000007,"5min":8.1630000000000003,"15min":7.2699999999999996},"max":{"1min":189.47900000000001,"5min":25434.151999999998,"15min":27883.419999999998},"last":17.448}]},{"osd":5,"last update":"Sat Jul 29 14:05:08 2023","interfaces":[{"interface":"back","average":{"1min":17.529,"5min":28.314,"15min":72.040999999999997},"min":{"1min":14.367000000000001,"5min":11.215,"15min":10.683},"max":{"1min":22.012,"5min":230.16800000000001,"15min":1265.123},"last":18.257999999999999},{"interface":"front","average":{"1min":12.343999999999999,"5min":25.687999999999999,"15min":76.659000000000006},"min":{"1min":9.609,"5min":7.71,"15min":7.5540000000000003},"max":{"1min":21.030999999999999,"5min":255.23699999999999,"15min":1234.9300000000001},"last":8.5440000000000005}]},{"osd":6,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":21.149999999999999,"15min":25.884},"min":{"1min":10.222,"5min":9.9740000000000002,"15min":7.8689999999999998},"max":{"1min":78.843000000000004,"5min":95.558999999999997,"15min":243.66499999999999},"last":12.518000000000001},{"interface":"front","average":{"1min":24.378,"5min":25.236999999999998,"15min":32.558},"min":{"1min":12.111000000000001,"5min":10.542,"15min":10.542},"max":{"1min":92.266999999999996,"5min":148.334,"15min":454.68799999999999},"last":19.045999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-29T14:05:45.746 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-29T14:05:45.746 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-29T14:05:45.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.860+0000 7f51d7a2f640 10 monclient: get_monmap_and_config 2023-07-29T14:05:45.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.860+0000 7f51d7a2f640 10 monclient: build_initial_monmap 2023-07-29T14:05:45.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.860+0000 7f51d7a2f640 10 monclient: monmap: 2023-07-29T14:05:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:45.861976+0000 2023-07-29T14:05:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:45.861976+0000 2023-07-29T14:05:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:45.871 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.861+0000 7f51d7a2f640 1 Processor -- start 2023-07-29T14:05:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.861+0000 7f51d7a2f640 1 -- start start 2023-07-29T14:05:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.861+0000 7f51d7a2f640 10 monclient: init 2023-07-29T14:05:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2023-07-29T14:05:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f51d00b0b90 0x7f51d0141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 10 monclient(hunting): picked mon.noname-c con 0x7f51d00b0b90 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51d00b0010 0x7f51d013baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 10 monclient(hunting): picked mon.noname-a con 0x7f51d00b0010 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f51d01423e0 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d00b0f60 con 0x7f51d01423e0 2023-07-29T14:05:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d00af430 con 0x7f51d00b0010 2023-07-29T14:05:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d00ae400 con 0x7f51d00b0b90 2023-07-29T14:05:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d7a2f640 10 monclient(hunting): authenticate will time out at 3667.995463s 2023-07-29T14:05:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d4fca640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51d00b0010 0x7f51d013baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d57cb640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f51d00b0b90 0x7f51d0141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d5fcc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f51d01423e0 0x7f51d0065920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51750/0 (socket says 172.21.15.40:51750) 2023-07-29T14:05:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d5fcc640 1 -- 172.21.15.40:0/1245064724 learned_addr learned my addr 172.21.15.40:0/1245064724 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d57cb640 10 monclient(hunting): get_auth_request con 0x7f51d00b0b90 auth_method 0 2023-07-29T14:05:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d57cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d57cb640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d57cb640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d4fca640 10 monclient(hunting): get_auth_request con 0x7f51d00b0010 auth_method 0 2023-07-29T14:05:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d4fca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d4fca640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.863+0000 7f51d4fca640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.864+0000 7f51d57cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.864+0000 7f51d57cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.864+0000 7f51d57cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.864+0000 7f51d4fca640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.864+0000 7f51d4fca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.864+0000 7f51d4fca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.865+0000 7f51d57cb640 10 monclient(hunting): handle_auth_done global_id 20338 payload 274 2023-07-29T14:05:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.865+0000 7f51d57cb640 1 -- 172.21.15.40:0/1245064724 >> v2:172.21.15.40:3300/0 conn(0x7f51d00b0010 msgr2=0x7f51d013baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.865+0000 7f51d57cb640 1 --2- 172.21.15.40:0/1245064724 >> v2:172.21.15.40:3300/0 conn(0x7f51d00b0010 0x7f51d013baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.865+0000 7f51d57cb640 1 -- 172.21.15.40:0/1245064724 >> v1:172.21.15.40:6789/0 conn(0x7f51d01423e0 legacy=0x7f51d0065920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.865+0000 7f51d57cb640 10 monclient: _finish_hunting 0 2023-07-29T14:05:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.865+0000 7f51d57cb640 1 monclient: found mon.noname-c 2023-07-29T14:05:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.865+0000 7f51d57cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.865+0000 7f51d57cb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T14:05:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.865+0000 7f51d57cb640 1 -- 172.21.15.40:0/1245064724 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d00ae730 con 0x7f51d00b0b90 2023-07-29T14:05:45.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.867+0000 7f51d57cb640 1 --2- 172.21.15.40:0/1245064724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d00b0b90 0x7f51d0141ea0 secure :-1 s=READY pgs=4780 cs=0 l=1 rev1=1 crypto rx=0x7f51d0066930 tx=0x7f51cc02be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69024c4387b249f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:45.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 1 -- 172.21.15.40:0/1245064724 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc02cbf0 con 0x7f51d00b0b90 2023-07-29T14:05:45.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:45.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:45.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 10 monclient: dump: 2023-07-29T14:05:45.887 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:45.887 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:45.888 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.888 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.888 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:45.888 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.888 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:45.888 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:45.889 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:45.889 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 10 monclient: _finish_auth 0 2023-07-29T14:05:45.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 10 monclient: _check_auth_tickets 2023-07-29T14:05:45.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:45.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 1 -- 172.21.15.40:0/1245064724 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc02cd50 con 0x7f51d00b0b90 2023-07-29T14:05:45.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:45.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 1 -- 172.21.15.40:0/1245064724 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc034660 con 0x7f51d00b0b90 2023-07-29T14:05:45.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:45.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T14:05:45.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.871+0000 7f51c7fff640 10 monclient: dump: 2023-07-29T14:05:45.891 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:45.891 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:45.891 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.891 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.891 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:45.891 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.891 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:45.892 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 5 monclient: authenticate success, global_id 20338 2023-07-29T14:05:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 4 set_mon_vals no callback set 2023-07-29T14:05:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 10 monclient: shutdown 2023-07-29T14:05:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 1 -- 172.21.15.40:0/1245064724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d00b0b90 msgr2=0x7f51d0141ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 1 --2- 172.21.15.40:0/1245064724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d00b0b90 0x7f51d0141ea0 secure :-1 s=READY pgs=4780 cs=0 l=1 rev1=1 crypto rx=0x7f51d0066930 tx=0x7f51cc02be40 comp rx=0 tx=0).stop 2023-07-29T14:05:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 1 -- 172.21.15.40:0/1245064724 shutdown_connections 2023-07-29T14:05:45.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 1 --2- 172.21.15.40:0/1245064724 >> v2:172.21.15.40:3300/0 conn(0x7f51d00b0010 0x7f51d013baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 1 --2- 172.21.15.40:0/1245064724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d00b0b90 0x7f51d0141ea0 unknown :-1 s=CLOSED pgs=4780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 1 -- 172.21.15.40:0/1245064724 >> 172.21.15.40:0/1245064724 conn(0x7f51d0069dc0 msgr2=0x7f51d006a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:45.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 1 -- 172.21.15.40:0/1245064724 shutdown_connections 2023-07-29T14:05:45.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.872+0000 7f51d7a2f640 1 -- 172.21.15.40:0/1245064724 wait complete. 2023-07-29T14:05:45.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.873+0000 7f51d7a2f640 10 monclient: build_initial_monmap 2023-07-29T14:05:45.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.873+0000 7f51d7a2f640 10 monclient: monmap: 2023-07-29T14:05:45.895 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:45.895 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:45.895 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:45.895 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:45.895 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:45.895 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.895 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:45.895 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:45.896 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.873+0000 7f51d7a2f640 1 librados: starting msgr at 2023-07-29T14:05:45.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.873+0000 7f51d7a2f640 1 librados: starting objecter 2023-07-29T14:05:45.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.873+0000 7f51d7a2f640 1 Processor -- start 2023-07-29T14:05:45.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.873+0000 7f51d7a2f640 1 -- start start 2023-07-29T14:05:45.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.873+0000 7f51d7a2f640 1 librados: setting wanted keys 2023-07-29T14:05:45.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.873+0000 7f51d7a2f640 1 librados: calling monclient init 2023-07-29T14:05:45.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.873+0000 7f51d7a2f640 10 monclient: init 2023-07-29T14:05:45.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:45.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T14:05:45.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f51d00b0010 0x7f51d006f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:45.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 10 monclient(hunting): picked mon.noname-c con 0x7f51d00b0010 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:45.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d00b0b90 0x7f51d00acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:45.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 10 monclient(hunting): picked mon.noname-a con 0x7f51d00b0b90 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:45.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d01423e0 0x7f51d006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:45.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 10 monclient(hunting): picked mon.noname-b con 0x7f51d01423e0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:45.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:45.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d011b8d0 con 0x7f51d00b0b90 2023-07-29T14:05:45.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:45.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d011d0c0 con 0x7f51d01423e0 2023-07-29T14:05:45.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:45.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d006fbb0 con 0x7f51d00b0010 2023-07-29T14:05:45.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:45.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d7a2f640 10 monclient(hunting): authenticate will time out at 3668.007544s 2023-07-29T14:05:45.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.874+0000 7f51d57cb640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f51d00b0010 0x7f51d006f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:45.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d4fca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d00b0b90 0x7f51d00acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:45.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d5fcc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d01423e0 0x7f51d006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d4fca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d00b0b90 0x7f51d00acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53822/0 (socket says 172.21.15.40:53822) 2023-07-29T14:05:45.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d4fca640 1 -- 172.21.15.40:0/3430841971 learned_addr learned my addr 172.21.15.40:0/3430841971 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:45.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d4fca640 10 monclient(hunting): get_auth_request con 0x7f51d00b0b90 auth_method 0 2023-07-29T14:05:45.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d4fca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:45.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d4fca640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:45.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d4fca640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:45.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d57cb640 10 monclient(hunting): get_auth_request con 0x7f51d00b0010 auth_method 0 2023-07-29T14:05:45.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d57cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:45.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d57cb640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:45.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d57cb640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:45.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d5fcc640 10 monclient(hunting): get_auth_request con 0x7f51d01423e0 auth_method 0 2023-07-29T14:05:45.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d5fcc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:45.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d5fcc640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:45.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d5fcc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:45.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d4fca640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:45.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d4fca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:45.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d4fca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:45.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d57cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:45.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d57cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:45.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d57cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:45.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d5fcc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:45.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d5fcc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:45.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.875+0000 7f51d5fcc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:45.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.878+0000 7f51d4fca640 10 monclient(hunting): handle_auth_done global_id 14898 payload 824 2023-07-29T14:05:45.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.878+0000 7f51d4fca640 1 -- 172.21.15.40:0/3430841971 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f51d00b0010 msgr2=0x7f51d006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:45.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.878+0000 7f51d4fca640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f51d00b0010 0x7f51d006f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.878+0000 7f51d4fca640 1 -- 172.21.15.40:0/3430841971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d01423e0 msgr2=0x7f51d006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:45.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.878+0000 7f51d4fca640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d01423e0 0x7f51d006dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:45.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.878+0000 7f51d4fca640 10 monclient: _finish_hunting 0 2023-07-29T14:05:45.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.878+0000 7f51d4fca640 1 monclient: found mon.noname-a 2023-07-29T14:05:45.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.878+0000 7f51d4fca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:45.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.878+0000 7f51d4fca640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T14:05:45.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.878+0000 7f51d4fca640 1 -- 172.21.15.40:0/3430841971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc009a70 con 0x7f51d00b0b90 2023-07-29T14:05:45.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.881+0000 7f51d4fca640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d00b0b90 0x7f51d00acd20 secure :-1 s=READY pgs=3208 cs=0 l=1 rev1=1 crypto rx=0x7f51c000e9b0 tx=0x7f51c000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:45.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.881+0000 7f51c5ffb640 10 client.?.objecter ms_handle_connect 0x7f51d00b0b90 2023-07-29T14:05:45.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.881+0000 7f51c5ffb640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:45.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 1 -- 172.21.15.40:0/3430841971 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0009670 con 0x7f51d00b0b90 2023-07-29T14:05:45.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:45.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T14:05:45.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 10 monclient: dump: 2023-07-29T14:05:45.925 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:45.925 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:45.925 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.926 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.926 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:45.926 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.926 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:45.926 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:45.926 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:45.926 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 10 monclient: _finish_auth 0 2023-07-29T14:05:45.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 10 monclient: _check_auth_tickets 2023-07-29T14:05:45.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:45.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 1 -- 172.21.15.40:0/3430841971 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c000c9a0 con 0x7f51d00b0b90 2023-07-29T14:05:45.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:45.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 1 -- 172.21.15.40:0/3430841971 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c000cd50 con 0x7f51d00b0b90 2023-07-29T14:05:45.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51d7a2f640 5 monclient: authenticate success, global_id 14898 2023-07-29T14:05:45.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51d7a2f640 10 monclient: _renew_subs 2023-07-29T14:05:45.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51d7a2f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:45.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51d7a2f640 1 -- 172.21.15.40:0/3430841971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d011bd60 con 0x7f51d00b0b90 2023-07-29T14:05:45.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c77fe640 4 set_mon_vals no callback set 2023-07-29T14:05:45.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:45.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:45.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51c5ffb640 10 monclient: dump: 2023-07-29T14:05:45.928 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:45.928 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:45.929 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.929 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:45.929 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:45.929 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:45.929 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:45.929 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:45.929 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:45.929 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:45.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51d7a2f640 10 client.14898.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:45.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51d7a2f640 10 monclient: _renew_subs 2023-07-29T14:05:45.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51d7a2f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:45.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51d7a2f640 1 -- 172.21.15.40:0/3430841971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d006e1a0 con 0x7f51d00b0b90 2023-07-29T14:05:45.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.885+0000 7f51d7a2f640 1 librados: init done 2023-07-29T14:05:45.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.886+0000 7f51d7a2f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:45.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.886+0000 7f51d7a2f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:45.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.886+0000 7f51d7a2f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:45.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:45.886+0000 7f51d7a2f640 1 -- 172.21.15.40:0/3430841971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d011d200 con 0x7f51d00b0b90 2023-07-29T14:05:46.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.142+0000 7f51c5ffb640 1 -- 172.21.15.40:0/3430841971 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f51c0013470 con 0x7f51d00b0b90 2023-07-29T14:05:46.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.143+0000 7f51c5ffb640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f51b406bfb0 0x7f51b406e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:46.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.143+0000 7f51d57cb640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f51b406bfb0 0x7f51b406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:46.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.143+0000 7f51d57cb640 10 monclient: get_auth_request con 0x7f51b406bfb0 auth_method 0 2023-07-29T14:05:46.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.144+0000 7f51d57cb640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f51b406bfb0 0x7f51b406e460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f51cc036000 tx=0x7f51cc030000 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:46.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.144+0000 7f51c5ffb640 10 client.14898.objecter ms_handle_connect 0x7f51b406bfb0 2023-07-29T14:05:46.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.155+0000 7f51c5ffb640 1 -- 172.21.15.40:0/3430841971 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f51c005c360 con 0x7f51d00b0b90 2023-07-29T14:05:46.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.155+0000 7f51c5ffb640 10 client.14898.objecter ms_dispatch 0x7f51d0000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:46.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.155+0000 7f51c5ffb640 3 client.14898.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:46.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.155+0000 7f51c5ffb640 3 client.14898.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:46.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.155+0000 7f51c5ffb640 20 client.14898.objecter dump_active .. 0 homeless 2023-07-29T14:05:46.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.155+0000 7f51c5ffb640 1 -- 172.21.15.40:0/3430841971 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f51c0054790 con 0x7f51d00b0b90 2023-07-29T14:05:46.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.155+0000 7f51c5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:46.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.155+0000 7f51c5ffb640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.336+0000 7f51d7a2f640 1 -- 172.21.15.40:0/3430841971 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51d0070d10 con 0x7f51b406bfb0 2023-07-29T14:05:46.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.341+0000 7f51c5ffb640 1 -- 172.21.15.40:0/3430841971 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152049 (secure 0 0 0) 0x7f51cc043070 con 0x7f51b406bfb0 2023-07-29T14:05:46.342 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T14:05:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-29T14:05:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 10 librados: watch_flush enter 2023-07-29T14:05:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 10 librados: watch_flush exit 2023-07-29T14:05:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 20 client.14898.objecter shutdown clearing up homeless session... 2023-07-29T14:05:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 10 client.14898.objecter successfully canceled tick 2023-07-29T14:05:46.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 -- 172.21.15.40:0/3430841971 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f51b406bfb0 msgr2=0x7f51b406e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:46.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f51b406bfb0 0x7f51b406e460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f51cc036000 tx=0x7f51cc030000 comp rx=0 tx=0).stop 2023-07-29T14:05:46.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 10 monclient: shutdown 2023-07-29T14:05:46.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:46.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 -- 172.21.15.40:0/3430841971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d00b0b90 msgr2=0x7f51d00acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:46.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d00b0b90 0x7f51d00acd20 secure :-1 s=READY pgs=3208 cs=0 l=1 rev1=1 crypto rx=0x7f51c000e9b0 tx=0x7f51c000ee70 comp rx=0 tx=0).stop 2023-07-29T14:05:46.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 -- 172.21.15.40:0/3430841971 shutdown_connections 2023-07-29T14:05:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f51b406bfb0 0x7f51b406e460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d01423e0 0x7f51d006dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d00b0b90 0x7f51d00acd20 unknown :-1 s=CLOSED pgs=3208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:46.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 --2- 172.21.15.40:0/3430841971 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f51d00b0010 0x7f51d006f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:46.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 -- 172.21.15.40:0/3430841971 >> 172.21.15.40:0/3430841971 conn(0x7f51d0069dc0 msgr2=0x7f51d0066070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:46.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 -- 172.21.15.40:0/3430841971 shutdown_connections 2023-07-29T14:05:46.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 -- 172.21.15.40:0/3430841971 wait complete. 2023-07-29T14:05:46.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.343+0000 7f51d7a2f640 1 librados: shutdown 2023-07-29T14:05:46.357 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":2134,"stamp":"2023-07-29T14:05:45.630383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":523924,"kb_used_data":50744,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754450796,"statfs":{"total":773094113280,"available":772557615104,"internally_reserved":0,"allocated":51961856,"data_stored":28370954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":132,"apply_latency_ms":132,"commit_latency_ns":132000000,"apply_latency_ns":132000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.154570"},"pg_stats":[{"pgid":"7.1b","version":"0'0","reported_seq":969,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.563113+0000","last_change":"2023-07-29T13:56:55.662556+0000","last_active":"2023-07-29T14:05:37.563113+0000","last_peered":"2023-07-29T14:05:37.563113+0000","last_clean":"2023-07-29T14:05:37.563113+0000","last_became_active":"2023-07-29T13:56:55.656930+0000","last_became_peered":"2023-07-29T13:56:55.656930+0000","last_unstale":"2023-07-29T14:05:37.563113+0000","last_undegraded":"2023-07-29T14:05:37.563113+0000","last_fullsized":"2023-07-29T14:05:37.563113+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:52:51.321547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":978,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.410164+0000","last_change":"2023-07-29T13:41:38.328763+0000","last_active":"2023-07-29T14:05:37.410164+0000","last_peered":"2023-07-29T14:05:37.410164+0000","last_clean":"2023-07-29T14:05:37.410164+0000","last_became_active":"2023-07-29T13:41:38.322679+0000","last_became_peered":"2023-07-29T13:41:38.322679+0000","last_unstale":"2023-07-29T14:05:37.410164+0000","last_undegraded":"2023-07-29T14:05:37.410164+0000","last_fullsized":"2023-07-29T14:05:37.410164+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:40:55.886733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":965,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.400523+0000","last_change":"2023-07-29T13:29:52.105383+0000","last_active":"2023-07-29T14:05:37.400523+0000","last_peered":"2023-07-29T14:05:37.400523+0000","last_clean":"2023-07-29T14:05:37.400523+0000","last_became_active":"2023-07-29T13:29:52.099163+0000","last_became_peered":"2023-07-29T13:29:52.099163+0000","last_unstale":"2023-07-29T14:05:37.400523+0000","last_undegraded":"2023-07-29T14:05:37.400523+0000","last_fullsized":"2023-07-29T14:05:37.400523+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:03:36.535099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":932,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.793552+0000","last_change":"2023-07-29T13:29:41.816664+0000","last_active":"2023-07-29T14:05:39.793552+0000","last_peered":"2023-07-29T14:05:39.793552+0000","last_clean":"2023-07-29T14:05:39.793552+0000","last_became_active":"2023-07-29T13:29:41.782506+0000","last_became_peered":"2023-07-29T13:29:41.782506+0000","last_unstale":"2023-07-29T14:05:39.793552+0000","last_undegraded":"2023-07-29T14:05:39.793552+0000","last_fullsized":"2023-07-29T14:05:39.793552+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:20:57.566653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":948,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.074570+0000","last_change":"2023-07-29T14:05:38.066403+0000","last_active":"2023-07-29T14:05:38.074570+0000","last_peered":"2023-07-29T14:05:38.074570+0000","last_clean":"2023-07-29T14:05:38.074570+0000","last_became_active":"2023-07-29T14:05:38.060277+0000","last_became_peered":"2023-07-29T14:05:38.060277+0000","last_unstale":"2023-07-29T14:05:38.074570+0000","last_undegraded":"2023-07-29T14:05:38.074570+0000","last_fullsized":"2023-07-29T14:05:38.074570+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:50:25.119138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":914,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.478515+0000","last_change":"2023-07-29T13:53:19.606247+0000","last_active":"2023-07-29T14:05:37.478515+0000","last_peered":"2023-07-29T14:05:37.478515+0000","last_clean":"2023-07-29T14:05:37.478515+0000","last_became_active":"2023-07-29T13:53:19.600386+0000","last_became_peered":"2023-07-29T13:53:19.600386+0000","last_unstale":"2023-07-29T14:05:37.478515+0000","last_undegraded":"2023-07-29T14:05:37.478515+0000","last_fullsized":"2023-07-29T14:05:37.478515+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:46:03.055229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":937,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.571721+0000","last_change":"2023-07-29T13:56:55.406251+0000","last_active":"2023-07-29T14:05:37.571721+0000","last_peered":"2023-07-29T14:05:37.571721+0000","last_clean":"2023-07-29T14:05:37.571721+0000","last_became_active":"2023-07-29T13:56:55.400591+0000","last_became_peered":"2023-07-29T13:56:55.400591+0000","last_unstale":"2023-07-29T14:05:37.571721+0000","last_undegraded":"2023-07-29T14:05:37.571721+0000","last_fullsized":"2023-07-29T14:05:37.571721+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:40:48.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":955,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.503487+0000","last_change":"2023-07-29T13:29:31.089676+0000","last_active":"2023-07-29T14:05:37.503487+0000","last_peered":"2023-07-29T14:05:37.503487+0000","last_clean":"2023-07-29T14:05:37.503487+0000","last_became_active":"2023-07-29T13:29:31.072042+0000","last_became_peered":"2023-07-29T13:29:31.072042+0000","last_unstale":"2023-07-29T14:05:37.503487+0000","last_undegraded":"2023-07-29T14:05:37.503487+0000","last_fullsized":"2023-07-29T14:05:37.503487+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:28:28.141003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":913,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.480494+0000","last_change":"2023-07-29T13:53:19.677067+0000","last_active":"2023-07-29T14:05:37.480494+0000","last_peered":"2023-07-29T14:05:37.480494+0000","last_clean":"2023-07-29T14:05:37.480494+0000","last_became_active":"2023-07-29T13:53:19.671215+0000","last_became_peered":"2023-07-29T13:53:19.671215+0000","last_unstale":"2023-07-29T14:05:37.480494+0000","last_undegraded":"2023-07-29T14:05:37.480494+0000","last_fullsized":"2023-07-29T14:05:37.480494+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:51:18.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":987,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.432372+0000","last_change":"2023-07-29T13:56:55.517147+0000","last_active":"2023-07-29T14:05:37.432372+0000","last_peered":"2023-07-29T14:05:37.432372+0000","last_clean":"2023-07-29T14:05:37.432372+0000","last_became_active":"2023-07-29T13:56:55.511490+0000","last_became_peered":"2023-07-29T13:56:55.511490+0000","last_unstale":"2023-07-29T14:05:37.432372+0000","last_undegraded":"2023-07-29T14:05:37.432372+0000","last_fullsized":"2023-07-29T14:05:37.432372+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:26.433868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":969,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.634817+0000","last_change":"2023-07-29T14:05:37.627583+0000","last_active":"2023-07-29T14:05:37.634817+0000","last_peered":"2023-07-29T14:05:37.634817+0000","last_clean":"2023-07-29T14:05:37.634817+0000","last_became_active":"2023-07-29T14:05:37.617822+0000","last_became_peered":"2023-07-29T14:05:37.617822+0000","last_unstale":"2023-07-29T14:05:37.634817+0000","last_undegraded":"2023-07-29T14:05:37.634817+0000","last_fullsized":"2023-07-29T14:05:37.634817+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:03:53.476296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":914,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.555235+0000","last_change":"2023-07-29T13:53:19.644608+0000","last_active":"2023-07-29T14:05:39.555235+0000","last_peered":"2023-07-29T14:05:39.555235+0000","last_clean":"2023-07-29T14:05:39.555235+0000","last_became_active":"2023-07-29T13:53:19.637473+0000","last_became_peered":"2023-07-29T13:53:19.637473+0000","last_unstale":"2023-07-29T14:05:39.555235+0000","last_undegraded":"2023-07-29T14:05:39.555235+0000","last_fullsized":"2023-07-29T14:05:39.555235+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:53:02.384899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":933,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.555513+0000","last_change":"2023-07-29T13:29:33.530600+0000","last_active":"2023-07-29T14:05:37.555513+0000","last_peered":"2023-07-29T14:05:37.555513+0000","last_clean":"2023-07-29T14:05:37.555513+0000","last_became_active":"2023-07-29T13:29:33.515255+0000","last_became_peered":"2023-07-29T13:29:33.515255+0000","last_unstale":"2023-07-29T14:05:37.555513+0000","last_undegraded":"2023-07-29T14:05:37.555513+0000","last_fullsized":"2023-07-29T14:05:37.555513+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:21:56.794070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":986,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.524744+0000","last_change":"2023-07-29T14:05:37.517334+0000","last_active":"2023-07-29T14:05:37.524744+0000","last_peered":"2023-07-29T14:05:37.524744+0000","last_clean":"2023-07-29T14:05:37.524744+0000","last_became_active":"2023-07-29T14:05:37.510437+0000","last_became_peered":"2023-07-29T14:05:37.510437+0000","last_unstale":"2023-07-29T14:05:37.524744+0000","last_undegraded":"2023-07-29T14:05:37.524744+0000","last_fullsized":"2023-07-29T14:05:37.524744+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:38:04.875711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":939,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.573766+0000","last_change":"2023-07-29T13:56:55.146603+0000","last_active":"2023-07-29T14:05:37.573766+0000","last_peered":"2023-07-29T14:05:37.573766+0000","last_clean":"2023-07-29T14:05:37.573766+0000","last_became_active":"2023-07-29T13:56:55.139167+0000","last_became_peered":"2023-07-29T13:56:55.139167+0000","last_unstale":"2023-07-29T14:05:37.573766+0000","last_undegraded":"2023-07-29T14:05:37.573766+0000","last_fullsized":"2023-07-29T14:05:37.573766+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:56:23.477235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":951,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.627838+0000","last_change":"2023-07-29T14:05:37.621178+0000","last_active":"2023-07-29T14:05:37.627838+0000","last_peered":"2023-07-29T14:05:37.627838+0000","last_clean":"2023-07-29T14:05:37.627838+0000","last_became_active":"2023-07-29T14:05:37.614802+0000","last_became_peered":"2023-07-29T14:05:37.614802+0000","last_unstale":"2023-07-29T14:05:37.627838+0000","last_undegraded":"2023-07-29T14:05:37.627838+0000","last_fullsized":"2023-07-29T14:05:37.627838+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:42:16.790121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":990,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.389647+0000","last_change":"2023-07-29T13:56:55.073408+0000","last_active":"2023-07-29T14:05:38.389647+0000","last_peered":"2023-07-29T14:05:38.389647+0000","last_clean":"2023-07-29T14:05:38.389647+0000","last_became_active":"2023-07-29T13:56:55.065724+0000","last_became_peered":"2023-07-29T13:56:55.065724+0000","last_unstale":"2023-07-29T14:05:38.389647+0000","last_undegraded":"2023-07-29T14:05:38.389647+0000","last_fullsized":"2023-07-29T14:05:38.389647+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:42:58.825948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":972,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.422249+0000","last_change":"2023-07-29T13:28:14.434312+0000","last_active":"2023-07-29T14:05:37.422249+0000","last_peered":"2023-07-29T14:05:37.422249+0000","last_clean":"2023-07-29T14:05:37.422249+0000","last_became_active":"2023-07-29T13:28:14.423044+0000","last_became_peered":"2023-07-29T13:28:14.423044+0000","last_unstale":"2023-07-29T14:05:37.422249+0000","last_undegraded":"2023-07-29T14:05:37.422249+0000","last_fullsized":"2023-07-29T14:05:37.422249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:56:27.865715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1","version":"70'7","reported_seq":984,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.521327+0000","last_change":"2023-07-29T13:41:38.528285+0000","last_active":"2023-07-29T14:05:37.521327+0000","last_peered":"2023-07-29T14:05:37.521327+0000","last_clean":"2023-07-29T14:05:37.521327+0000","last_became_active":"2023-07-29T13:41:38.521847+0000","last_became_peered":"2023-07-29T13:41:38.521847+0000","last_unstale":"2023-07-29T14:05:37.521327+0000","last_undegraded":"2023-07-29T14:05:37.521327+0000","last_fullsized":"2023-07-29T14:05:37.521327+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:08:42.836378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":945,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.566657+0000","last_change":"2023-07-29T13:56:55.263959+0000","last_active":"2023-07-29T14:05:37.566657+0000","last_peered":"2023-07-29T14:05:37.566657+0000","last_clean":"2023-07-29T14:05:37.566657+0000","last_became_active":"2023-07-29T13:56:55.258241+0000","last_became_peered":"2023-07-29T13:56:55.258241+0000","last_unstale":"2023-07-29T14:05:37.566657+0000","last_undegraded":"2023-07-29T14:05:37.566657+0000","last_fullsized":"2023-07-29T14:05:37.566657+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:42.517469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":967,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.437586+0000","last_change":"2023-07-29T13:29:52.230014+0000","last_active":"2023-07-29T14:05:37.437586+0000","last_peered":"2023-07-29T14:05:37.437586+0000","last_clean":"2023-07-29T14:05:37.437586+0000","last_became_active":"2023-07-29T13:29:52.224467+0000","last_became_peered":"2023-07-29T13:29:52.224467+0000","last_unstale":"2023-07-29T14:05:37.437586+0000","last_undegraded":"2023-07-29T14:05:37.437586+0000","last_fullsized":"2023-07-29T14:05:37.437586+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:51:03.019114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":974,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.636002+0000","last_change":"2023-07-29T13:28:14.652586+0000","last_active":"2023-07-29T14:05:39.636002+0000","last_peered":"2023-07-29T14:05:39.636002+0000","last_clean":"2023-07-29T14:05:39.636002+0000","last_became_active":"2023-07-29T13:28:14.642023+0000","last_became_peered":"2023-07-29T13:28:14.642023+0000","last_unstale":"2023-07-29T14:05:39.636002+0000","last_undegraded":"2023-07-29T14:05:39.636002+0000","last_fullsized":"2023-07-29T14:05:39.636002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:25:23.334956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":934,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.566809+0000","last_change":"2023-07-29T13:29:41.869711+0000","last_active":"2023-07-29T14:05:37.566809+0000","last_peered":"2023-07-29T14:05:37.566809+0000","last_clean":"2023-07-29T14:05:37.566809+0000","last_became_active":"2023-07-29T13:29:41.844314+0000","last_became_peered":"2023-07-29T13:29:41.844314+0000","last_unstale":"2023-07-29T14:05:37.566809+0000","last_undegraded":"2023-07-29T14:05:37.566809+0000","last_fullsized":"2023-07-29T14:05:37.566809+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:41:40.503015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":964,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.476599+0000","last_change":"2023-07-29T13:56:55.159956+0000","last_active":"2023-07-29T14:05:37.476599+0000","last_peered":"2023-07-29T14:05:37.476599+0000","last_clean":"2023-07-29T14:05:37.476599+0000","last_became_active":"2023-07-29T13:56:55.150514+0000","last_became_peered":"2023-07-29T13:56:55.150514+0000","last_unstale":"2023-07-29T14:05:37.476599+0000","last_undegraded":"2023-07-29T14:05:37.476599+0000","last_fullsized":"2023-07-29T14:05:37.476599+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:16:38.329000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1006,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.420320+0000","last_change":"2023-07-29T13:56:55.615681+0000","last_active":"2023-07-29T14:05:37.420320+0000","last_peered":"2023-07-29T14:05:37.420320+0000","last_clean":"2023-07-29T14:05:37.420320+0000","last_became_active":"2023-07-29T13:56:55.609414+0000","last_became_peered":"2023-07-29T13:56:55.609414+0000","last_unstale":"2023-07-29T14:05:37.420320+0000","last_undegraded":"2023-07-29T14:05:37.420320+0000","last_fullsized":"2023-07-29T14:05:37.420320+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:36:30.601005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1013,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.391985+0000","last_change":"2023-07-29T13:56:55.550112+0000","last_active":"2023-07-29T14:05:37.391985+0000","last_peered":"2023-07-29T14:05:37.391985+0000","last_clean":"2023-07-29T14:05:37.391985+0000","last_became_active":"2023-07-29T13:56:55.543861+0000","last_became_peered":"2023-07-29T13:56:55.543861+0000","last_unstale":"2023-07-29T14:05:37.391985+0000","last_undegraded":"2023-07-29T14:05:37.391985+0000","last_fullsized":"2023-07-29T14:05:37.391985+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:57:36.179121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":951,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.620825+0000","last_change":"2023-07-29T14:05:37.613527+0000","last_active":"2023-07-29T14:05:37.620825+0000","last_peered":"2023-07-29T14:05:37.620825+0000","last_clean":"2023-07-29T14:05:37.620825+0000","last_became_active":"2023-07-29T14:05:37.607023+0000","last_became_peered":"2023-07-29T14:05:37.607023+0000","last_unstale":"2023-07-29T14:05:37.620825+0000","last_undegraded":"2023-07-29T14:05:37.620825+0000","last_fullsized":"2023-07-29T14:05:37.620825+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:07:39.130065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":974,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.659241+0000","last_change":"2023-07-29T13:28:14.713485+0000","last_active":"2023-07-29T14:05:39.659241+0000","last_peered":"2023-07-29T14:05:39.659241+0000","last_clean":"2023-07-29T14:05:39.659241+0000","last_became_active":"2023-07-29T13:28:14.708042+0000","last_became_peered":"2023-07-29T13:28:14.708042+0000","last_unstale":"2023-07-29T14:05:39.659241+0000","last_undegraded":"2023-07-29T14:05:39.659241+0000","last_fullsized":"2023-07-29T14:05:39.659241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:25:56.525512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":1102,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.391897+0000","last_change":"2023-07-29T13:56:55.164457+0000","last_active":"2023-07-29T14:05:38.391897+0000","last_peered":"2023-07-29T14:05:38.391897+0000","last_clean":"2023-07-29T14:05:38.391897+0000","last_became_active":"2023-07-29T13:56:55.157702+0000","last_became_peered":"2023-07-29T13:56:55.157702+0000","last_unstale":"2023-07-29T14:05:38.391897+0000","last_undegraded":"2023-07-29T14:05:38.391897+0000","last_fullsized":"2023-07-29T14:05:38.391897+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:06.434639+0000","last_clean_scrub_stamp":"2023-07-29T13:28:06.434639+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:06:43.352094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":996,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.500941+0000","last_change":"2023-07-29T13:56:55.347022+0000","last_active":"2023-07-29T14:05:37.500941+0000","last_peered":"2023-07-29T14:05:37.500941+0000","last_clean":"2023-07-29T14:05:37.500941+0000","last_became_active":"2023-07-29T13:56:55.341426+0000","last_became_peered":"2023-07-29T13:56:55.341426+0000","last_unstale":"2023-07-29T14:05:37.500941+0000","last_undegraded":"2023-07-29T14:05:37.500941+0000","last_fullsized":"2023-07-29T14:05:37.500941+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:46:20.874037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":947,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.437114+0000","last_change":"2023-07-29T13:41:38.063359+0000","last_active":"2023-07-29T14:05:37.437114+0000","last_peered":"2023-07-29T14:05:37.437114+0000","last_clean":"2023-07-29T14:05:37.437114+0000","last_became_active":"2023-07-29T13:41:38.057649+0000","last_became_peered":"2023-07-29T13:41:38.057649+0000","last_unstale":"2023-07-29T14:05:37.437114+0000","last_undegraded":"2023-07-29T14:05:37.437114+0000","last_fullsized":"2023-07-29T14:05:37.437114+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:36:46.868780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":926,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.484786+0000","last_change":"2023-07-29T13:53:19.566408+0000","last_active":"2023-07-29T14:05:37.484786+0000","last_peered":"2023-07-29T14:05:37.484786+0000","last_clean":"2023-07-29T14:05:37.484786+0000","last_became_active":"2023-07-29T13:53:19.560480+0000","last_became_peered":"2023-07-29T13:53:19.560480+0000","last_unstale":"2023-07-29T14:05:37.484786+0000","last_undegraded":"2023-07-29T14:05:37.484786+0000","last_fullsized":"2023-07-29T14:05:37.484786+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:01:49.170231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":914,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.488215+0000","last_change":"2023-07-29T13:53:19.617225+0000","last_active":"2023-07-29T14:05:37.488215+0000","last_peered":"2023-07-29T14:05:37.488215+0000","last_clean":"2023-07-29T14:05:37.488215+0000","last_became_active":"2023-07-29T13:53:19.611477+0000","last_became_peered":"2023-07-29T13:53:19.611477+0000","last_unstale":"2023-07-29T14:05:37.488215+0000","last_undegraded":"2023-07-29T14:05:37.488215+0000","last_fullsized":"2023-07-29T14:05:37.488215+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:52:56.727655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":951,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.539929+0000","last_change":"2023-07-29T13:56:55.182600+0000","last_active":"2023-07-29T14:05:37.539929+0000","last_peered":"2023-07-29T14:05:37.539929+0000","last_clean":"2023-07-29T14:05:37.539929+0000","last_became_active":"2023-07-29T13:56:55.173228+0000","last_became_peered":"2023-07-29T13:56:55.173228+0000","last_unstale":"2023-07-29T14:05:37.539929+0000","last_undegraded":"2023-07-29T14:05:37.539929+0000","last_fullsized":"2023-07-29T14:05:37.539929+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:57:08.005958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":938,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.548791+0000","last_change":"2023-07-29T13:56:55.243737+0000","last_active":"2023-07-29T14:05:37.548791+0000","last_peered":"2023-07-29T14:05:37.548791+0000","last_clean":"2023-07-29T14:05:37.548791+0000","last_became_active":"2023-07-29T13:56:55.238036+0000","last_became_peered":"2023-07-29T13:56:55.238036+0000","last_unstale":"2023-07-29T14:05:37.548791+0000","last_undegraded":"2023-07-29T14:05:37.548791+0000","last_fullsized":"2023-07-29T14:05:37.548791+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:04:27.494041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":932,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.553875+0000","last_change":"2023-07-29T13:29:33.527250+0000","last_active":"2023-07-29T14:05:37.553875+0000","last_peered":"2023-07-29T14:05:37.553875+0000","last_clean":"2023-07-29T14:05:37.553875+0000","last_became_active":"2023-07-29T13:29:33.508113+0000","last_became_peered":"2023-07-29T13:29:33.508113+0000","last_unstale":"2023-07-29T14:05:37.553875+0000","last_undegraded":"2023-07-29T14:05:37.553875+0000","last_fullsized":"2023-07-29T14:05:37.553875+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T13:52:25.906655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":941,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.401769+0000","last_change":"2023-07-29T13:29:32.261353+0000","last_active":"2023-07-29T14:05:38.401769+0000","last_peered":"2023-07-29T14:05:38.401769+0000","last_clean":"2023-07-29T14:05:38.401769+0000","last_became_active":"2023-07-29T13:29:32.123788+0000","last_became_peered":"2023-07-29T13:29:32.123788+0000","last_unstale":"2023-07-29T14:05:38.401769+0000","last_undegraded":"2023-07-29T14:05:38.401769+0000","last_fullsized":"2023-07-29T14:05:38.401769+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:25:55.926424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":937,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.564990+0000","last_change":"2023-07-29T13:56:55.597264+0000","last_active":"2023-07-29T14:05:37.564990+0000","last_peered":"2023-07-29T14:05:37.564990+0000","last_clean":"2023-07-29T14:05:37.564990+0000","last_became_active":"2023-07-29T13:56:55.591631+0000","last_became_peered":"2023-07-29T13:56:55.591631+0000","last_unstale":"2023-07-29T14:05:37.564990+0000","last_undegraded":"2023-07-29T14:05:37.564990+0000","last_fullsized":"2023-07-29T14:05:37.564990+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:55:19.929908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1004,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.418634+0000","last_change":"2023-07-29T13:56:55.572081+0000","last_active":"2023-07-29T14:05:37.418634+0000","last_peered":"2023-07-29T14:05:37.418634+0000","last_clean":"2023-07-29T14:05:37.418634+0000","last_became_active":"2023-07-29T13:56:55.564155+0000","last_became_peered":"2023-07-29T13:56:55.564155+0000","last_unstale":"2023-07-29T14:05:37.418634+0000","last_undegraded":"2023-07-29T14:05:37.418634+0000","last_fullsized":"2023-07-29T14:05:37.418634+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:56:05.370398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":973,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.441262+0000","last_change":"2023-07-29T13:53:19.441193+0000","last_active":"2023-07-29T14:05:37.441262+0000","last_peered":"2023-07-29T14:05:37.441262+0000","last_clean":"2023-07-29T14:05:37.441262+0000","last_became_active":"2023-07-29T13:53:19.433950+0000","last_became_peered":"2023-07-29T13:53:19.433950+0000","last_unstale":"2023-07-29T14:05:37.441262+0000","last_undegraded":"2023-07-29T14:05:37.441262+0000","last_fullsized":"2023-07-29T14:05:37.441262+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:38:39.350563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":959,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.530325+0000","last_change":"2023-07-29T14:05:37.522505+0000","last_active":"2023-07-29T14:05:37.530325+0000","last_peered":"2023-07-29T14:05:37.530325+0000","last_clean":"2023-07-29T14:05:37.530325+0000","last_became_active":"2023-07-29T14:05:37.513918+0000","last_became_peered":"2023-07-29T14:05:37.513918+0000","last_unstale":"2023-07-29T14:05:37.530325+0000","last_undegraded":"2023-07-29T14:05:37.530325+0000","last_fullsized":"2023-07-29T14:05:37.530325+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:34:20.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":17,"seq":73014444473,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59672,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312168,"statfs":{"total":96636764160,"available":96575660032,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":15.343999999999999,"5min":16.634,"15min":170.43700000000001},"min":{"1min":6.3490000000000002,"5min":6.3330000000000002,"15min":4.7590000000000003},"max":{"1min":111.66,"5min":172.44399999999999,"15min":12751.485000000001},"last":8.4399999999999995},{"interface":"front","average":{"1min":16.254000000000001,"5min":16.774000000000001,"15min":173.22},"min":{"1min":6.79,"5min":5.4500000000000002,"15min":5.4500000000000002},"max":{"1min":129.28399999999999,"5min":173.48699999999999,"15min":12747.76},"last":9.4369999999999994}]},{"osd":1,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":18.536000000000001,"5min":1543.261,"15min":525.65800000000002},"min":{"1min":7.0010000000000003,"5min":6.5149999999999997,"15min":6.5149999999999997},"max":{"1min":99.557000000000002,"5min":29038.089,"15min":29038.089},"last":12.125},{"interface":"front","average":{"1min":22.518000000000001,"5min":1548.174,"15min":530.21500000000003},"min":{"1min":6.9089999999999998,"5min":6.4749999999999996,"15min":6.2249999999999996},"max":{"1min":102.327,"5min":29039.032999999999,"15min":29039.032999999999},"last":38.445}]},{"osd":2,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":23.297000000000001,"5min":21.439,"15min":20.452000000000002},"min":{"1min":7.992,"5min":7.3200000000000003,"15min":7.3200000000000003},"max":{"1min":104.566,"5min":249.84800000000001,"15min":784.66200000000003},"last":39.115000000000002},{"interface":"front","average":{"1min":21.670000000000002,"5min":21.084,"15min":19.474},"min":{"1min":8.6280000000000001,"5min":8.3919999999999995,"15min":7.8049999999999997},"max":{"1min":99.706000000000003,"5min":202.64500000000001,"15min":368.62200000000001},"last":11.403}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.91,"5min":25.608000000000001,"15min":478.83800000000002},"min":{"1min":7.0910000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"max":{"1min":101.714,"5min":312.73599999999999,"15min":28381.707999999999},"last":12.787000000000001},{"interface":"front","average":{"1min":20.298999999999999,"5min":27.803999999999998,"15min":474.06299999999999},"min":{"1min":9.3219999999999992,"5min":9.3219999999999992,"15min":8.7520000000000007},"max":{"1min":97.727999999999994,"5min":242.63,"15min":28126.012999999999},"last":30.170000000000002}]},{"osd":5,"last update":"Sat Jul 29 14:05:01 2023","interfaces":[{"interface":"back","average":{"1min":17.222999999999999,"5min":20.141999999999999,"15min":32.993000000000002},"min":{"1min":10.452,"5min":8.4540000000000006,"15min":7.3689999999999998},"max":{"1min":79.867000000000004,"5min":85.578999999999994,"15min":488.72300000000001},"last":10.744},{"interface":"front","average":{"1min":14.161,"5min":19.236999999999998,"15min":21.838000000000001},"min":{"1min":8.1379999999999999,"5min":8.1379999999999999,"15min":7.3840000000000003},"max":{"1min":52.622,"5min":106.634,"15min":128.791},"last":30.861000000000001}]},{"osd":6,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":24.904,"5min":20.061,"15min":22.629999999999999},"min":{"1min":9.1470000000000002,"5min":8.0129999999999999,"15min":7.7779999999999996},"max":{"1min":106.33199999999999,"5min":124.819,"15min":787.18499999999995},"last":14.055},{"interface":"front","average":{"1min":25.198,"5min":20.042000000000002,"15min":22.454999999999998},"min":{"1min":9.8010000000000002,"5min":8.5790000000000006,"15min":8.3170000000000002},"max":{"1min":112.328,"5min":112.328,"15min":646.87599999999998},"last":31.472999999999999}]},{"osd":7,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":27.215,"5min":22.292000000000002,"15min":48.816000000000003},"min":{"1min":11.487,"5min":9.2620000000000005,"15min":9.2620000000000005},"max":{"1min":149.43199999999999,"5min":149.43199999999999,"15min":648.54200000000003},"last":32.210999999999999},{"interface":"front","average":{"1min":23.321000000000002,"5min":20.568999999999999,"15min":38.241999999999997},"min":{"1min":10.385,"5min":9.2300000000000004,"15min":8.7309999999999999},"max":{"1min":75.954999999999998,"5min":75.954999999999998,"15min":364.86500000000001},"last":37.286999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444485,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64136,"kb_used_data":6248,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94307704,"statfs":{"total":96636764160,"available":96571088896,"internally_reserved":0,"allocated":6397952,"data_stored":3446913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":13.247999999999999,"15min":130.82599999999999},"min":{"1min":5.3789999999999996,"5min":5.0209999999999999,"15min":4.476},"max":{"1min":29.634,"5min":93.590999999999994,"15min":10009.493},"last":11.063000000000001},{"interface":"front","average":{"1min":13.936999999999999,"5min":13.526,"15min":131.76599999999999},"min":{"1min":7.1299999999999999,"5min":6.2069999999999999,"15min":5.617},"max":{"1min":37.527999999999999,"5min":99.156999999999996,"15min":9995.5869999999995},"last":16.388000000000002}]},{"osd":1,"last update":"Sat Jul 29 14:04:47 2023","interfaces":[{"interface":"back","average":{"1min":11.255000000000001,"5min":1413.5239999999999,"15min":481.31099999999998},"min":{"1min":6.0899999999999999,"5min":6.0899999999999999,"15min":6.0899999999999999},"max":{"1min":17.850999999999999,"5min":29343.018,"15min":29343.018},"last":15.468999999999999},{"interface":"front","average":{"1min":11.234999999999999,"5min":1411.289,"15min":478.77199999999999},"min":{"1min":7.0209999999999999,"5min":6.5880000000000001,"15min":5.8860000000000001},"max":{"1min":18.347000000000001,"5min":29342.263999999999,"15min":29342.263999999999},"last":7.7919999999999998}]},{"osd":3,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":17.623000000000001,"5min":20.777000000000001,"15min":365.04399999999998},"min":{"1min":6.7809999999999997,"5min":6.7809999999999997,"15min":6.7809999999999997},"max":{"1min":86.926000000000002,"5min":331.92399999999998,"15min":29770.313999999998},"last":9.6850000000000005},{"interface":"front","average":{"1min":21.684999999999999,"5min":22.039999999999999,"15min":358.54500000000002},"min":{"1min":9.3070000000000004,"5min":8.0540000000000003,"15min":6.4749999999999996},"max":{"1min":89.156000000000006,"5min":332.49900000000002,"15min":29586.387999999999},"last":17.302}]},{"osd":4,"last update":"Sat Jul 29 14:04:53 2023","interfaces":[{"interface":"back","average":{"1min":12.471,"5min":819.34000000000003,"15min":870.13199999999995},"min":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"max":{"1min":52.584000000000003,"5min":25685.629000000001,"15min":32071.659},"last":11.618},{"interface":"front","average":{"1min":15.125999999999999,"5min":814.09699999999998,"15min":870.976},"min":{"1min":8.8330000000000002,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":35.020000000000003,"5min":25610.863000000001,"15min":32126.150000000001},"last":18.202999999999999}]},{"osd":5,"last update":"Sat Jul 29 14:05:06 2023","interfaces":[{"interface":"back","average":{"1min":14.398,"5min":20.428999999999998,"15min":38.106999999999999},"min":{"1min":8.7579999999999991,"5min":8.4269999999999996,"15min":8.4269999999999996},"max":{"1min":18.667000000000002,"5min":193.45400000000001,"15min":428.04000000000002},"last":10.387},{"interface":"front","average":{"1min":12.019,"5min":20.446999999999999,"15min":41.822000000000003},"min":{"1min":6.891,"5min":6.891,"15min":6.891},"max":{"1min":19.634,"5min":174.292,"15min":430.15499999999997},"last":12.253}]},{"osd":6,"last update":"Sat Jul 29 14:05:40 2023","interfaces":[{"interface":"back","average":{"1min":17.140000000000001,"5min":21.350999999999999,"15min":20.024000000000001},"min":{"1min":8.3450000000000006,"5min":8.3450000000000006,"15min":8.3450000000000006},"max":{"1min":77.855000000000004,"5min":318.45299999999997,"15min":318.45299999999997},"last":12.85},{"interface":"front","average":{"1min":17.119,"5min":21.058,"15min":20.518999999999998},"min":{"1min":9.2230000000000008,"5min":9.2230000000000008,"15min":7.5300000000000002},"max":{"1min":44.755000000000003,"5min":349.34899999999999,"15min":349.34899999999999},"last":20.885999999999999}]},{"osd":7,"last update":"Sat Jul 29 14:04:42 2023","interfaces":[{"interface":"back","average":{"1min":15.279999999999999,"5min":20.536000000000001,"15min":21.027000000000001},"min":{"1min":10.663,"5min":8.5079999999999991,"15min":7.5890000000000004},"max":{"1min":20.757999999999999,"5min":291.80900000000003,"15min":291.80900000000003},"last":13.475},{"interface":"front","average":{"1min":15.644,"5min":21.513000000000002,"15min":22.518000000000001},"min":{"1min":9.6419999999999995,"5min":9.1940000000000008,"15min":9.1940000000000008},"max":{"1min":23.408000000000001,"5min":291.44200000000001,"15min":291.44200000000001},"last":19.187000000000001}]}]},{"osd":1,"up_from":447,"seq":1919850381743,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60432,"kb_used_data":6640,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94311408,"statfs":{"total":96636764160,"available":96574881792,"internally_reserved":0,"allocated":6799360,"data_stored":3844753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":20.931000000000001,"5min":20.015999999999998,"15min":112.807},"min":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.3680000000000003},"max":{"1min":116.512,"5min":364.47800000000001,"15min":12525.57},"last":12.721},{"interface":"front","average":{"1min":18.943000000000001,"5min":20.664000000000001,"15min":112.874},"min":{"1min":6.0970000000000004,"5min":6.0970000000000004,"15min":4.6820000000000004},"max":{"1min":100.402,"5min":372.00999999999999,"15min":12523.974},"last":11.795999999999999}]},{"osd":2,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.987,"5min":26.809000000000001,"15min":19.399999999999999},"min":{"1min":7.819,"5min":6.5110000000000001,"15min":5.5919999999999996},"max":{"1min":69.497,"5min":360.88099999999997,"15min":360.88099999999997},"last":13.550000000000001},{"interface":"front","average":{"1min":19.43,"5min":28.155999999999999,"15min":20.445},"min":{"1min":7.093,"5min":7.093,"15min":6.2729999999999997},"max":{"1min":93.792000000000002,"5min":368.51400000000001,"15min":368.51400000000001},"last":17.792999999999999}]},{"osd":3,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":26.696000000000002,"5min":30.559000000000001,"15min":531.69799999999998},"min":{"1min":9.1590000000000007,"5min":6.7140000000000004,"15min":6.7140000000000004},"max":{"1min":216.16300000000001,"5min":435.92200000000003,"15min":34504.942000000003},"last":14.794},{"interface":"front","average":{"1min":31.332000000000001,"5min":41.100999999999999,"15min":539.97299999999996},"min":{"1min":10.423,"5min":10.423,"15min":5.3639999999999999},"max":{"1min":238.33699999999999,"5min":700.10500000000002,"15min":34620.970999999998},"last":21.510999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.427,"5min":1170.991,"15min":980.98599999999999},"min":{"1min":8.4130000000000003,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":50.289000000000001,"5min":25247.098999999998,"15min":32258.529999999999},"last":16.050999999999998},{"interface":"front","average":{"1min":16.536000000000001,"5min":1165.587,"15min":967.39999999999998},"min":{"1min":11.340999999999999,"5min":8.9169999999999998,"15min":6.5090000000000003},"max":{"1min":29.376999999999999,"5min":25174.008999999998,"15min":31933.992999999999},"last":19.478999999999999}]},{"osd":5,"last update":"Sat Jul 29 14:05:13 2023","interfaces":[{"interface":"back","average":{"1min":15.249000000000001,"5min":33.526000000000003,"15min":43.616},"min":{"1min":10.416,"5min":10.294,"15min":10.294},"max":{"1min":23.649999999999999,"5min":433.79300000000001,"15min":433.79300000000001},"last":22.321999999999999},{"interface":"front","average":{"1min":15.952,"5min":36.515000000000001,"15min":46.286000000000001},"min":{"1min":11.467000000000001,"5min":9.6929999999999996,"15min":8.7260000000000009},"max":{"1min":22.300999999999998,"5min":419.22800000000001,"15min":419.22800000000001},"last":18.899999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":16.931999999999999,"5min":21.992000000000001,"15min":19.459},"min":{"1min":6.6079999999999997,"5min":6.327,"15min":6.327},"max":{"1min":154.92400000000001,"5min":357.20499999999998,"15min":357.20499999999998},"last":9.1189999999999998},{"interface":"front","average":{"1min":20.251999999999999,"5min":27.460999999999999,"15min":23.411000000000001},"min":{"1min":10.877000000000001,"5min":9.5079999999999991,"15min":9.5079999999999991},"max":{"1min":155.994,"5min":359.68099999999998,"15min":359.68099999999998},"last":16.553999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.683,"5min":35.137999999999998,"15min":23.725999999999999},"min":{"1min":10.130000000000001,"5min":8.5199999999999996,"15min":7.5919999999999996},"max":{"1min":92.168999999999997,"5min":563.29700000000003,"15min":563.29700000000003},"last":20.231000000000002},{"interface":"front","average":{"1min":17.949000000000002,"5min":30.442,"15min":21.349},"min":{"1min":8.2240000000000002,"5min":8.2240000000000002,"15min":7.2919999999999998},"max":{"1min":113.15900000000001,"5min":352.65199999999999,"15min":352.65199999999999},"last":10.521000000000001}]}]},{"osd":0,"up_from":1002,"seq":4303557231029,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59676,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312164,"statfs":{"total":96636764160,"available":96575655936,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":71.099999999999994,"5min":1023.201,"15min":351.10899999999998},"min":{"1min":8.9689999999999994,"5min":5.9249999999999998,"15min":4.6269999999999998},"max":{"1min":350.00200000000001,"5min":27534,"15min":27534},"last":11.109999999999999},{"interface":"front","average":{"1min":99.875,"5min":1025.7080000000001,"15min":350.98899999999998},"min":{"1min":5.9589999999999996,"5min":5.7240000000000002,"15min":5.0739999999999998},"max":{"1min":591.69600000000003,"5min":27535.008999999998,"15min":27535.008999999998},"last":12.922000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.170999999999999,"5min":22.285,"15min":18.638000000000002},"min":{"1min":11.680999999999999,"5min":8.1069999999999993,"15min":6.6159999999999997},"max":{"1min":364.62799999999999,"5min":364.62799999999999,"15min":364.62799999999999},"last":8.9559999999999995},{"interface":"front","average":{"1min":45.890000000000001,"5min":26.417999999999999,"15min":21.443000000000001},"min":{"1min":14.172000000000001,"5min":10.023999999999999,"15min":8.8780000000000001},"max":{"1min":370.34300000000002,"5min":370.34300000000002,"15min":370.34300000000002},"last":15.214}]},{"osd":3,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":66.832999999999998,"5min":25.178000000000001,"15min":355.89100000000002},"min":{"1min":6.7610000000000001,"5min":6.4409999999999998,"15min":5.891},"max":{"1min":444.59500000000003,"5min":444.59500000000003,"15min":30170.040000000001},"last":12.173},{"interface":"front","average":{"1min":65.754000000000005,"5min":33.606000000000002,"15min":363.07400000000001},"min":{"1min":10.266999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"max":{"1min":386.06099999999998,"5min":386.06099999999998,"15min":30171.194},"last":16.274999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":92.555000000000007,"5min":36.014000000000003,"15min":585.43899999999996},"min":{"1min":12.201000000000001,"5min":6.4480000000000004,"15min":6.0979999999999999},"max":{"1min":445.65300000000002,"5min":445.65300000000002,"15min":29892.669999999998},"last":19.030999999999999},{"interface":"front","average":{"1min":115.806,"5min":40.560000000000002,"15min":588.88900000000001},"min":{"1min":7.923,"5min":5.9400000000000004,"15min":5.9400000000000004},"max":{"1min":593.13499999999999,"5min":593.13499999999999,"15min":29877.289000000001},"last":9.3350000000000009}]},{"osd":5,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.341000000000001,"5min":28.972000000000001,"15min":56.932000000000002},"min":{"1min":12.702,"5min":9.5700000000000003,"15min":9.5700000000000003},"max":{"1min":403.13999999999999,"5min":403.13999999999999,"15min":508.02100000000002},"last":17.283000000000001},{"interface":"front","average":{"1min":43.966999999999999,"5min":23.216000000000001,"15min":32.374000000000002},"min":{"1min":7.6429999999999998,"5min":7.1210000000000004,"15min":7.1210000000000004},"max":{"1min":590.58500000000004,"5min":590.58500000000004,"15min":590.58500000000004},"last":17.960000000000001}]},{"osd":6,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":52.365000000000002,"5min":30.853999999999999,"15min":25.227},"min":{"1min":16.876000000000001,"5min":10.409000000000001,"15min":9.9350000000000005},"max":{"1min":375.36500000000001,"5min":375.36500000000001,"15min":375.36500000000001},"last":11.582000000000001},{"interface":"front","average":{"1min":55.634,"5min":32.579999999999998,"15min":27.417999999999999},"min":{"1min":18.984999999999999,"5min":11.564,"15min":11.564},"max":{"1min":380.31599999999997,"5min":380.31599999999997,"15min":380.31599999999997},"last":19.946000000000002}]},{"osd":7,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":57.960999999999999,"5min":28.638000000000002,"15min":20.780000000000001},"min":{"1min":17.748999999999999,"5min":8.4939999999999998,"15min":7.9349999999999996},"max":{"1min":385.01900000000001,"5min":385.01900000000001,"15min":385.01900000000001},"last":20.901},{"interface":"front","average":{"1min":54.853999999999999,"5min":29.765999999999998,"15min":25.545000000000002},"min":{"1min":10.992000000000001,"5min":8.1229999999999993,"15min":8.1229999999999993},"max":{"1min":446.69099999999997,"5min":446.69099999999997,"15min":446.69099999999997},"last":14.324999999999999}]}]},{"osd":4,"up_from":16,"seq":68719477162,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59676,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312164,"statfs":{"total":96636764160,"available":96575655936,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":13.432,"5min":21.457000000000001,"15min":87.253},"min":{"1min":6.4029999999999996,"5min":4.6230000000000002,"15min":4.6230000000000002},"max":{"1min":48.048999999999999,"5min":495.86900000000003,"15min":10627.831},"last":10.673},{"interface":"front","average":{"1min":18.010000000000002,"5min":25.013000000000002,"15min":90.852000000000004},"min":{"1min":7.2400000000000002,"5min":6.0990000000000002,"15min":5.6660000000000004},"max":{"1min":124.64100000000001,"5min":446.84100000000001,"15min":10653.877},"last":7.1440000000000001}]},{"osd":1,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":11.276999999999999,"5min":1441.567,"15min":495.98200000000003},"min":{"1min":7.2640000000000002,"5min":7.2640000000000002,"15min":5.9909999999999997},"max":{"1min":21.369,"5min":26872.857,"15min":26872.857},"last":12.961},{"interface":"front","average":{"1min":11.013,"5min":1440.0119999999999,"15min":493.40499999999997},"min":{"1min":7.4269999999999996,"5min":7.3090000000000002,"15min":5.6909999999999998},"max":{"1min":17.440999999999999,"5min":26874.048999999999,"15min":26874.048999999999},"last":16.699999999999999}]},{"osd":2,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":12.445,"5min":24.050999999999998,"15min":23.050000000000001},"min":{"1min":8.9000000000000004,"5min":7.3019999999999996,"15min":7.3019999999999996},"max":{"1min":15.904999999999999,"5min":453.40100000000001,"15min":597.11800000000005},"last":13.585000000000001},{"interface":"front","average":{"1min":15.029999999999999,"5min":27.077999999999999,"15min":26.128},"min":{"1min":12.429,"5min":11.631,"15min":8.0559999999999992},"max":{"1min":19.111000000000001,"5min":453.96199999999999,"15min":705.04499999999996},"last":14.6}]},{"osd":3,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":14.789999999999999,"5min":27.626999999999999,"15min":590.51199999999994},"min":{"1min":9.7219999999999995,"5min":7.6799999999999997,"15min":5.9740000000000002},"max":{"1min":23.942,"5min":448.65100000000001,"15min":30725.612000000001},"last":15.250999999999999},{"interface":"front","average":{"1min":12.497,"5min":20.047999999999998,"15min":582.06899999999996},"min":{"1min":7.7690000000000001,"5min":6.3840000000000003,"15min":6.3840000000000003},"max":{"1min":19.815000000000001,"5min":336.62599999999998,"15min":30407.659},"last":17.337}]},{"osd":5,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":16.190999999999999,"5min":30.579000000000001,"15min":32.869},"min":{"1min":14.1,"5min":7.1440000000000001,"15min":7.1440000000000001},"max":{"1min":18.738,"5min":450.81700000000001,"15min":450.81700000000001},"last":15.920999999999999},{"interface":"front","average":{"1min":10.430999999999999,"5min":19.959,"15min":22.803999999999998},"min":{"1min":6.2779999999999996,"5min":6.2779999999999996,"15min":6.2779999999999996},"max":{"1min":20.254999999999999,"5min":340.04000000000002,"15min":340.04000000000002},"last":9.093}]},{"osd":6,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":13.007,"5min":22.312000000000001,"15min":22.866},"min":{"1min":8.1120000000000001,"5min":7.1689999999999996,"15min":6.1520000000000001},"max":{"1min":22.010000000000002,"5min":339.47199999999998,"15min":696.952},"last":9.7759999999999998},{"interface":"front","average":{"1min":14.724,"5min":23.917999999999999,"15min":23.209},"min":{"1min":9.0289999999999999,"5min":6.3449999999999998,"15min":6.3449999999999998},"max":{"1min":20.968,"5min":498.26999999999998,"15min":598.86900000000003},"last":19.806000000000001}]},{"osd":7,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":12.805,"5min":24.097000000000001,"15min":24.802},"min":{"1min":8.7430000000000003,"5min":8.3399999999999999,"15min":7.7930000000000001},"max":{"1min":18.638000000000002,"5min":340.61900000000003,"15min":699.31799999999998},"last":18.100999999999999},{"interface":"front","average":{"1min":17.483000000000001,"5min":31.318999999999999,"15min":33.040999999999997},"min":{"1min":13.861000000000001,"5min":11.067,"15min":11.067},"max":{"1min":19.817,"5min":455.339,"15min":703.68200000000002},"last":11.962}]}]},{"osd":6,"up_from":15,"seq":64424509874,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82180,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94289660,"statfs":{"total":96636764160,"available":96552611840,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":12.891999999999999,"5min":16.984000000000002,"15min":62.423000000000002},"min":{"1min":6.9390000000000001,"5min":5.2350000000000003,"15min":5.2350000000000003},"max":{"1min":29.821000000000002,"5min":224.93899999999999,"15min":9502.5869999999995},"last":5.5629999999999997},{"interface":"front","average":{"1min":15.925000000000001,"5min":19.867000000000001,"15min":65.921999999999997},"min":{"1min":8.2050000000000001,"5min":6.1580000000000004,"15min":6.1580000000000004},"max":{"1min":31.303999999999998,"5min":130.48400000000001,"15min":9511.3719999999994},"last":9.6059999999999999}]},{"osd":1,"last update":"Sat Jul 29 14:04:49 2023","interfaces":[{"interface":"back","average":{"1min":17.140999999999998,"5min":970.20899999999995,"15min":335.46300000000002},"min":{"1min":7.5839999999999996,"5min":7.3369999999999997,"15min":4.8529999999999998},"max":{"1min":73.679000000000002,"5min":26903.921999999999,"15min":26903.921999999999},"last":14.039999999999999},{"interface":"front","average":{"1min":12.878,"5min":963.24699999999996,"15min":332.12599999999998},"min":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.3330000000000002},"max":{"1min":69.432000000000002,"5min":26893.022000000001,"15min":26893.022000000001},"last":11.784000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":14.891999999999999,"5min":21.295999999999999,"15min":19.027000000000001},"min":{"1min":10.173,"5min":7.8609999999999998,"15min":6.6299999999999999},"max":{"1min":26.302,"5min":191.73099999999999,"15min":191.73099999999999},"last":15.919},{"interface":"front","average":{"1min":11.848000000000001,"5min":16.003,"15min":15.422000000000001},"min":{"1min":7.4690000000000003,"5min":5.5499999999999998,"15min":5.5499999999999998},"max":{"1min":18,"5min":189.40199999999999,"15min":189.40199999999999},"last":10.874000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":10.653,"5min":15.525,"15min":521.48800000000006},"min":{"1min":7.2619999999999996,"5min":6.335,"15min":6.335},"max":{"1min":23.292000000000002,"5min":187.852,"15min":30710.619999999999},"last":12.523999999999999},{"interface":"front","average":{"1min":15.202999999999999,"5min":17.899000000000001,"15min":521.02300000000002},"min":{"1min":9.907,"5min":7.7009999999999996,"15min":7.6180000000000003},"max":{"1min":40.335000000000001,"5min":101.88200000000001,"15min":30591.776999999998},"last":17.108000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:04:56 2023","interfaces":[{"interface":"back","average":{"1min":16.030000000000001,"5min":808.61300000000006,"15min":1019.547},"min":{"1min":8.0950000000000006,"5min":6.718,"15min":6.1070000000000002},"max":{"1min":73.055999999999997,"5min":25620.438999999998,"15min":32899.582000000002},"last":8.8569999999999993},{"interface":"front","average":{"1min":16.128,"5min":828.35799999999995,"15min":1009.853},"min":{"1min":10.254,"5min":6.2089999999999996,"15min":6.2089999999999996},"max":{"1min":40.786999999999999,"5min":25979.712,"15min":32485.278999999999},"last":15.188000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:05:04 2023","interfaces":[{"interface":"back","average":{"1min":17.489999999999998,"5min":21.376999999999999,"15min":24.847000000000001},"min":{"1min":11.52,"5min":9.4949999999999992,"15min":8.9290000000000003},"max":{"1min":34.677,"5min":75.25,"15min":189.423},"last":53.837000000000003},{"interface":"front","average":{"1min":14.324,"5min":18.073,"15min":21.690000000000001},"min":{"1min":8.6969999999999992,"5min":7.3330000000000002,"15min":7.3330000000000002},"max":{"1min":24.77,"5min":80.085999999999999,"15min":188.518},"last":52.890000000000001}]},{"osd":7,"last update":"Sat Jul 29 14:04:41 2023","interfaces":[{"interface":"back","average":{"1min":24.321000000000002,"5min":19.489999999999998,"15min":19.814},"min":{"1min":14.164,"5min":7.1059999999999999,"15min":7.1059999999999999},"max":{"1min":74.337000000000003,"5min":74.337000000000003,"15min":200.55099999999999},"last":19.123000000000001},{"interface":"front","average":{"1min":18.581,"5min":20.780000000000001,"15min":22.265000000000001},"min":{"1min":9.6880000000000006,"5min":8.0790000000000006,"15min":8.0790000000000006},"max":{"1min":47.313000000000002,"5min":89.125,"15min":222.65199999999999},"last":17.780999999999999}]}]},{"osd":5,"up_from":704,"seq":3023656976773,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78476,"kb_used_data":6636,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94293364,"statfs":{"total":96636764160,"available":96556404736,"internally_reserved":0,"allocated":6795264,"data_stored":3844748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:04:00 2023","interfaces":[{"interface":"back","average":{"1min":23.48,"5min":22.623000000000001,"15min":41.000999999999998},"min":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":4.742},"max":{"1min":150.63,"5min":262.56099999999998,"15min":299.87799999999999},"last":8.3529999999999998},{"interface":"front","average":{"1min":24.837,"5min":22.727,"15min":52.545999999999999},"min":{"1min":5.8259999999999996,"5min":4.6539999999999999,"15min":4.6539999999999999},"max":{"1min":111.14100000000001,"5min":253.80000000000001,"15min":370.00999999999999},"last":7.3300000000000001}]},{"osd":1,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":13.867000000000001,"5min":25.277000000000001,"15min":33.274999999999999},"min":{"1min":6.5419999999999998,"5min":5.3780000000000001,"15min":5.3780000000000001},"max":{"1min":91.590000000000003,"5min":362.58800000000002,"15min":362.58800000000002},"last":12.525},{"interface":"front","average":{"1min":15.465999999999999,"5min":34.079000000000001,"15min":56.872999999999998},"min":{"1min":7.5650000000000004,"5min":6.8680000000000003,"15min":6.8680000000000003},"max":{"1min":85.760000000000005,"5min":356.69900000000001,"15min":363.50999999999999},"last":9.1530000000000005}]},{"osd":2,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.152000000000001,"5min":26.193000000000001,"15min":51.703000000000003},"min":{"1min":8.1880000000000006,"5min":7.032,"15min":7.032},"max":{"1min":151.804,"5min":312.755,"15min":384.024},"last":18.709},{"interface":"front","average":{"1min":16.754000000000001,"5min":20.221,"15min":25.251000000000001},"min":{"1min":7.782,"5min":7.2759999999999998,"15min":7.2709999999999999},"max":{"1min":102.09099999999999,"5min":228.55600000000001,"15min":228.55600000000001},"last":14.336}]},{"osd":3,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":28.802,"15min":55.957999999999998},"min":{"1min":10.734,"5min":7.4100000000000001,"15min":7.4100000000000001},"max":{"1min":131.36600000000001,"5min":312.12200000000001,"15min":381.23200000000003},"last":14.715},{"interface":"front","average":{"1min":18.568999999999999,"5min":28.875,"15min":66.894999999999996},"min":{"1min":7.944,"5min":7.944,"15min":7.944},"max":{"1min":118.456,"5min":265.92700000000002,"15min":382.779},"last":13.188000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":19.385999999999999,"5min":1108.5630000000001,"15min":412.19600000000003},"min":{"1min":10.5,"5min":7.343,"15min":7.3070000000000004},"max":{"1min":103.768,"5min":23572.741999999998,"15min":23572.741999999998},"last":16.469999999999999},{"interface":"front","average":{"1min":20.039000000000001,"5min":1113.885,"15min":428.73500000000001},"min":{"1min":8.7260000000000009,"5min":6.8929999999999998,"15min":6.8929999999999998},"max":{"1min":137.57400000000001,"5min":23610.142,"15min":23610.142},"last":9.8550000000000004}]},{"osd":6,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":17.074000000000002,"5min":25.704999999999998,"15min":51.972000000000001},"min":{"1min":9.7360000000000007,"5min":8.1709999999999994,"15min":8.1709999999999994},"max":{"1min":41.347000000000001,"5min":263.983,"15min":450.17500000000001},"last":11.878},{"interface":"front","average":{"1min":18.436,"5min":26.684999999999999,"15min":57.575000000000003},"min":{"1min":11.185,"5min":8.1969999999999992,"15min":8.1969999999999992},"max":{"1min":63.707999999999998,"5min":221.99199999999999,"15min":417.303},"last":17.875}]},{"osd":7,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":21.550999999999998,"5min":28.064,"15min":47.530999999999999},"min":{"1min":12.340999999999999,"5min":11.222,"15min":10.602},"max":{"1min":104.718,"5min":284.48000000000002,"15min":341.30500000000001},"last":19.625},{"interface":"front","average":{"1min":18.460999999999999,"5min":27.806000000000001,"15min":53.993000000000002},"min":{"1min":12.089,"5min":11.042999999999999,"15min":11.042999999999999},"max":{"1min":46.353999999999999,"5min":191.33099999999999,"15min":329.86500000000001},"last":15.612}]}]},{"osd":7,"up_from":650,"seq":2791728742801,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59676,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312164,"statfs":{"total":96636764160,"available":96575655936,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:45 2023","interfaces":[{"interface":"back","average":{"1min":11.94,"5min":13.635,"15min":23.015000000000001},"min":{"1min":5.5890000000000004,"5min":5.4539999999999997,"15min":5.4539999999999997},"max":{"1min":28.5,"5min":75.968000000000004,"15min":293.94499999999999},"last":10.49},{"interface":"front","average":{"1min":14.968999999999999,"5min":18.658999999999999,"15min":28.620000000000001},"min":{"1min":6.8179999999999996,"5min":6.8179999999999996,"15min":6.8179999999999996},"max":{"1min":29.143999999999998,"5min":116.952,"15min":358.267},"last":9.1709999999999994}]},{"osd":1,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":19.048999999999999,"5min":1108.855,"15min":389.67099999999999},"min":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":4.6660000000000004},"max":{"1min":184.25800000000001,"5min":27546.963,"15min":27546.963},"last":11.231999999999999},{"interface":"front","average":{"1min":22.969999999999999,"5min":1111.8620000000001,"15min":385.10300000000001},"min":{"1min":8.6110000000000007,"5min":5.5010000000000003,"15min":5.5010000000000003},"max":{"1min":150.75700000000001,"5min":27546.280999999999,"15min":27546.280999999999},"last":9.7629999999999999}]},{"osd":2,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.088000000000001,"5min":19.381,"15min":28.02},"min":{"1min":10.154999999999999,"5min":7.9100000000000001,"15min":7.9100000000000001},"max":{"1min":103.09,"5min":103.09,"15min":461.83800000000002},"last":15.819000000000001},{"interface":"front","average":{"1min":21.068000000000001,"5min":15.704000000000001,"15min":23.861999999999998},"min":{"1min":7.1269999999999998,"5min":6.6239999999999997,"15min":6.6239999999999997},"max":{"1min":181.97900000000001,"5min":181.97900000000001,"15min":385.40800000000002},"last":7.0039999999999996}]},{"osd":3,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":25.065999999999999,"5min":21.029,"15min":1531.4280000000001},"min":{"1min":7.976,"5min":7.6660000000000004,"15min":7.6660000000000004},"max":{"1min":242.197,"5min":242.197,"15min":34550.838000000003},"last":7.8300000000000001},{"interface":"front","average":{"1min":31.428999999999998,"5min":27.873000000000001,"15min":1543.2929999999999},"min":{"1min":12.592000000000001,"5min":6.9550000000000001,"15min":6.9550000000000001},"max":{"1min":240.46899999999999,"5min":243.06399999999999,"15min":34556.381000000001},"last":16.466999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":20.402999999999999,"5min":1074.817,"15min":900.84500000000003},"min":{"1min":6.7220000000000004,"5min":6.3719999999999999,"15min":6.3719999999999999},"max":{"1min":103.931,"5min":25146.41,"15min":27891.68},"last":11.835000000000001},{"interface":"front","average":{"1min":29.045999999999999,"5min":1099.6300000000001,"15min":918.40200000000004},"min":{"1min":9.7360000000000007,"5min":8.1630000000000003,"15min":7.2699999999999996},"max":{"1min":189.47900000000001,"5min":25434.151999999998,"15min":27883.419999999998},"last":17.448}]},{"osd":5,"last update":"Sat Jul 29 14:05:08 2023","interfaces":[{"interface":"back","average":{"1min":17.529,"5min":28.314,"15min":72.040999999999997},"min":{"1min":14.367000000000001,"5min":11.215,"15min":10.683},"max":{"1min":22.012,"5min":230.16800000000001,"15min":1265.123},"last":18.257999999999999},{"interface":"front","average":{"1min":12.343999999999999,"5min":25.687999999999999,"15min":76.659000000000006},"min":{"1min":9.609,"5min":7.71,"15min":7.5540000000000003},"max":{"1min":21.030999999999999,"5min":255.23699999999999,"15min":1234.9300000000001},"last":8.5440000000000005}]},{"osd":6,"last update":"Sat Jul 29 14:04:46 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":21.149999999999999,"15min":25.884},"min":{"1min":10.222,"5min":9.9740000000000002,"15min":7.8689999999999998},"max":{"1min":78.843000000000004,"5min":95.558999999999997,"15min":243.66499999999999},"last":12.518000000000001},{"interface":"front","average":{"1min":24.378,"5min":25.236999999999998,"15min":32.558},"min":{"1min":12.111000000000001,"5min":10.542,"15min":10.542},"max":{"1min":92.266999999999996,"5min":148.334,"15min":454.68799999999999},"last":19.045999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-29T14:05:46.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-29T14:05:46.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.471+0000 7efcc46ce640 10 monclient: get_monmap_and_config 2023-07-29T14:05:46.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.471+0000 7efcc46ce640 10 monclient: build_initial_monmap 2023-07-29T14:05:46.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.471+0000 7efcc46ce640 10 monclient: monmap: 2023-07-29T14:05:46.490 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:46.490 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:46.490 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:46.471490+0000 2023-07-29T14:05:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:46.471490+0000 2023-07-29T14:05:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:46.492 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.471+0000 7efcc46ce640 1 Processor -- start 2023-07-29T14:05:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.471+0000 7efcc46ce640 1 -- start start 2023-07-29T14:05:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.471+0000 7efcc46ce640 10 monclient: init 2023-07-29T14:05:46.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:46.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2023-07-29T14:05:46.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efcbc0b3ae0 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:46.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efcbc13e3a0 0x7efcbc140c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:46.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 10 monclient(hunting): picked mon.noname-c con 0x7efcbc13e3a0 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:46.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efcbc0a8910 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:46.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc0b3eb0 con 0x7efcbc0a8910 2023-07-29T14:05:46.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc0a9950 con 0x7efcbc0b3ae0 2023-07-29T14:05:46.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:46.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc0b0f10 con 0x7efcbc13e3a0 2023-07-29T14:05:46.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:46.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.472+0000 7efcc46ce640 10 monclient(hunting): authenticate will time out at 3668.604782s 2023-07-29T14:05:46.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.473+0000 7efcc2c6b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efcbc0a8910 0x7efcbc14b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51758/0 (socket says 172.21.15.40:51758) 2023-07-29T14:05:46.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.473+0000 7efcc2c6b640 1 -- 172.21.15.40:0/3657401299 learned_addr learned my addr 172.21.15.40:0/3657401299 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:46.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.473+0000 7efcc1c69640 1 --2- 172.21.15.40:0/3657401299 >> v2:172.21.15.45:3300/0 conn(0x7efcbc13e3a0 0x7efcbc140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.474+0000 7efcc1c69640 10 monclient(hunting): get_auth_request con 0x7efcbc13e3a0 auth_method 0 2023-07-29T14:05:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.474+0000 7efcc1c69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.474+0000 7efcc1c69640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.474+0000 7efcc1c69640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.474+0000 7efcc1c69640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.474+0000 7efcc1c69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.474+0000 7efcc1c69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.475+0000 7efcc1468640 1 -- 172.21.15.40:0/3657401299 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270135049 0 0) 0x7efcbc0b3eb0 con 0x7efcbc0a8910 2023-07-29T14:05:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.475+0000 7efcc1468640 10 monclient(hunting): _negotiate 2023-07-29T14:05:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.475+0000 7efcc1468640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.475+0000 7efcc1468640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:46.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.475+0000 7efcc1468640 10 monclient(hunting): my global_id is 14901 2023-07-29T14:05:46.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.475+0000 7efcc1468640 1 -- 172.21.15.40:0/3657401299 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efca8003520 con 0x7efcbc0a8910 2023-07-29T14:05:46.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.476+0000 7efcc1c69640 10 monclient(hunting): handle_auth_done global_id 20344 payload 274 2023-07-29T14:05:46.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.476+0000 7efcc1c69640 1 -- 172.21.15.40:0/3657401299 >> v1:172.21.15.45:6789/0 conn(0x7efcbc0b3ae0 legacy=0x7efcbc13dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:46.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.476+0000 7efcc1c69640 1 -- 172.21.15.40:0/3657401299 >> v1:172.21.15.40:6789/0 conn(0x7efcbc0a8910 legacy=0x7efcbc14b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:46.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.476+0000 7efcc1c69640 10 monclient: _finish_hunting 0 2023-07-29T14:05:46.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.476+0000 7efcc1c69640 1 monclient: found mon.noname-c 2023-07-29T14:05:46.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.476+0000 7efcc1c69640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:46.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.476+0000 7efcc1c69640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T14:05:46.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.476+0000 7efcc1c69640 1 -- 172.21.15.40:0/3657401299 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc0a9c30 con 0x7efcbc13e3a0 2023-07-29T14:05:46.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.479+0000 7efcc1c69640 1 --2- 172.21.15.40:0/3657401299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcbc13e3a0 0x7efcbc140c10 secure :-1 s=READY pgs=4781 cs=0 l=1 rev1=1 crypto rx=0x7efcac0097a0 tx=0x7efcac02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e125ff698a15d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:46.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.482+0000 7efcc1468640 1 -- 172.21.15.40:0/3657401299 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac02e070 con 0x7efcbc13e3a0 2023-07-29T14:05:46.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.482+0000 7efcc1468640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:46.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.482+0000 7efcc1468640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:46.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.482+0000 7efcc1468640 10 monclient: dump: 2023-07-29T14:05:46.511 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:46.511 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:46.511 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:46.511 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:46.511 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:46.512 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:46.512 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:46.512 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:46.512 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:46.512 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:46.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.482+0000 7efcc1468640 10 monclient: _finish_auth 0 2023-07-29T14:05:46.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.482+0000 7efcc1468640 10 monclient: _check_auth_tickets 2023-07-29T14:05:46.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.482+0000 7efcc1468640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:46.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.482+0000 7efcc46ce640 5 monclient: authenticate success, global_id 20344 2023-07-29T14:05:46.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.482+0000 7efcc46ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T14:05:46.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.484+0000 7efcc1468640 1 -- 172.21.15.40:0/3657401299 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac0315a0 con 0x7efcbc13e3a0 2023-07-29T14:05:46.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.484+0000 7efcc1468640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:46.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.484+0000 7efcc1468640 1 -- 172.21.15.40:0/3657401299 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac031910 con 0x7efcbc13e3a0 2023-07-29T14:05:46.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.484+0000 7efcc46ce640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:46.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.484+0000 7efcc46ce640 4 set_mon_vals no callback set 2023-07-29T14:05:46.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.484+0000 7efcc1468640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:46.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.484+0000 7efcc1468640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T14:05:46.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.484+0000 7efcc1468640 10 monclient: dump: 2023-07-29T14:05:46.523 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:46.523 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:46.524 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:46.524 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:46.524 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:46.525 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:46.525 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:46.525 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:46.525 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:46.526 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:46.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.485+0000 7efcc46ce640 10 monclient: shutdown 2023-07-29T14:05:46.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.485+0000 7efcc46ce640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:46.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.485+0000 7efcc46ce640 1 -- 172.21.15.40:0/3657401299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcbc13e3a0 msgr2=0x7efcbc140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:46.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.485+0000 7efcc46ce640 1 --2- 172.21.15.40:0/3657401299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcbc13e3a0 0x7efcbc140c10 secure :-1 s=READY pgs=4781 cs=0 l=1 rev1=1 crypto rx=0x7efcac0097a0 tx=0x7efcac02b840 comp rx=0 tx=0).stop 2023-07-29T14:05:46.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.485+0000 7efcc46ce640 1 -- 172.21.15.40:0/3657401299 shutdown_connections 2023-07-29T14:05:46.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.485+0000 7efcc46ce640 1 --2- 172.21.15.40:0/3657401299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcbc13e3a0 0x7efcbc140c10 unknown :-1 s=CLOSED pgs=4781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:46.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.485+0000 7efcc46ce640 1 -- 172.21.15.40:0/3657401299 >> 172.21.15.40:0/3657401299 conn(0x7efcbc070fc0 msgr2=0x7efcbc0713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:46.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.485+0000 7efcc46ce640 1 -- 172.21.15.40:0/3657401299 shutdown_connections 2023-07-29T14:05:46.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.485+0000 7efcc46ce640 1 -- 172.21.15.40:0/3657401299 wait complete. 2023-07-29T14:05:46.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient: build_initial_monmap 2023-07-29T14:05:46.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient: monmap: 2023-07-29T14:05:46.529 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:46.530 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:46.531 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:46.531 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:46.531 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:46.532 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:46.532 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:46.532 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:46.532 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:46.532 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:46.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 librados: starting msgr at 2023-07-29T14:05:46.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 librados: starting objecter 2023-07-29T14:05:46.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 Processor -- start 2023-07-29T14:05:46.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 -- start start 2023-07-29T14:05:46.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 librados: setting wanted keys 2023-07-29T14:05:46.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 librados: calling monclient init 2023-07-29T14:05:46.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient: init 2023-07-29T14:05:46.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:46.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T14:05:46.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcbc0a8910 0x7efcbc1d9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:46.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient(hunting): picked mon.noname-b con 0x7efcbc0a8910 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:46.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efcbc0b3ae0 0x7efcbc137f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:46.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient(hunting): picked mon.noname-c con 0x7efcbc0b3ae0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:46.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcbc13e3a0 0x7efcbc1384c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:46.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient(hunting): picked mon.noname-a con 0x7efcbc13e3a0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:46.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:46.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc1d9bf0 con 0x7efcbc13e3a0 2023-07-29T14:05:46.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:46.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc138a00 con 0x7efcbc0a8910 2023-07-29T14:05:46.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:46.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc138b70 con 0x7efcbc0b3ae0 2023-07-29T14:05:46.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:46.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.486+0000 7efcc46ce640 10 monclient(hunting): authenticate will time out at 3668.619072s 2023-07-29T14:05:46.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc2c6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcbc13e3a0 0x7efcbc1384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:46.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc1c69640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efcbc0b3ae0 0x7efcbc137f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:46.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc246a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcbc0a8910 0x7efcbc1d9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:46.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc2c6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcbc13e3a0 0x7efcbc1384c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53826/0 (socket says 172.21.15.40:53826) 2023-07-29T14:05:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc1c69640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efcbc0b3ae0 0x7efcbc137f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49448/0 (socket says 172.21.15.40:49448) 2023-07-29T14:05:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc2c6b640 1 -- 172.21.15.40:0/1274205621 learned_addr learned my addr 172.21.15.40:0/1274205621 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc2c6b640 10 monclient(hunting): get_auth_request con 0x7efcbc13e3a0 auth_method 0 2023-07-29T14:05:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc2c6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc2c6b640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc2c6b640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc1c69640 10 monclient(hunting): get_auth_request con 0x7efcbc0b3ae0 auth_method 0 2023-07-29T14:05:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc1c69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:46.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc1c69640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:46.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.487+0000 7efcc1c69640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:46.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc246a640 10 monclient(hunting): get_auth_request con 0x7efcbc0a8910 auth_method 0 2023-07-29T14:05:46.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc246a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:46.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc246a640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:46.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc246a640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:46.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc2c6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:46.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc2c6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:46.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc2c6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:46.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc1c69640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:46.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc1c69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:46.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc1c69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:46.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc246a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:46.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc246a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:46.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.488+0000 7efcc246a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:46.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.491+0000 7efcc2c6b640 10 monclient(hunting): handle_auth_done global_id 14904 payload 824 2023-07-29T14:05:46.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.491+0000 7efcc2c6b640 1 -- 172.21.15.40:0/1274205621 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efcbc0b3ae0 msgr2=0x7efcbc137f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:46.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.491+0000 7efcc2c6b640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efcbc0b3ae0 0x7efcbc137f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:46.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.491+0000 7efcc2c6b640 1 -- 172.21.15.40:0/1274205621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcbc0a8910 msgr2=0x7efcbc1d9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:46.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.491+0000 7efcc2c6b640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcbc0a8910 0x7efcbc1d9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:46.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.491+0000 7efcc2c6b640 10 monclient: _finish_hunting 0 2023-07-29T14:05:46.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.491+0000 7efcc2c6b640 1 monclient: found mon.noname-a 2023-07-29T14:05:46.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.491+0000 7efcc2c6b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:46.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.491+0000 7efcc2c6b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T14:05:46.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.491+0000 7efcc2c6b640 1 -- 172.21.15.40:0/1274205621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac009400 con 0x7efcbc13e3a0 2023-07-29T14:05:46.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.493+0000 7efcc2c6b640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcbc13e3a0 0x7efcbc1384c0 secure :-1 s=READY pgs=3210 cs=0 l=1 rev1=1 crypto rx=0x7efcb400cb30 tx=0x7efcb400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:46.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.493+0000 7efcb2ffd640 10 client.?.objecter ms_handle_connect 0x7efcbc13e3a0 2023-07-29T14:05:46.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.493+0000 7efcb2ffd640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:46.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 1 -- 172.21.15.40:0/1274205621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb400aca0 con 0x7efcbc13e3a0 2023-07-29T14:05:46.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:46.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T14:05:46.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 10 monclient: dump: 2023-07-29T14:05:46.542 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:46.542 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:46.542 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:46.542 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:46.542 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:46.543 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:46.543 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:46.543 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:46.543 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:46.543 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:46.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 10 monclient: _finish_auth 0 2023-07-29T14:05:46.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:46.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:46.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 1 -- 172.21.15.40:0/1274205621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb400ae00 con 0x7efcbc13e3a0 2023-07-29T14:05:46.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcc46ce640 5 monclient: authenticate success, global_id 14904 2023-07-29T14:05:46.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcc46ce640 10 monclient: _renew_subs 2023-07-29T14:05:46.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcc46ce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:46.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcc46ce640 1 -- 172.21.15.40:0/1274205621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc141a40 con 0x7efcbc13e3a0 2023-07-29T14:05:46.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:46.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 1 -- 172.21.15.40:0/1274205621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4012600 con 0x7efcbc13e3a0 2023-07-29T14:05:46.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:46.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:46.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcb2ffd640 10 monclient: dump: 2023-07-29T14:05:46.545 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:46.545 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:46.545 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:46.545 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:46.545 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:46.546 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:46.546 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:46.546 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:46.546 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:46.546 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:46.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcc0c67640 4 set_mon_vals no callback set 2023-07-29T14:05:46.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcc46ce640 10 client.14904.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:46.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcc46ce640 10 monclient: _renew_subs 2023-07-29T14:05:46.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcc46ce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:46.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcc46ce640 1 -- 172.21.15.40:0/1274205621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc1d96b0 con 0x7efcbc13e3a0 2023-07-29T14:05:46.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.497+0000 7efcc46ce640 1 librados: init done 2023-07-29T14:05:46.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.498+0000 7efcc46ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:46.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.498+0000 7efcc46ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:46.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.498+0000 7efcc46ce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:46.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.498+0000 7efcc46ce640 1 -- 172.21.15.40:0/1274205621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc0b4450 con 0x7efcbc13e3a0 2023-07-29T14:05:46.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.758+0000 7efcb2ffd640 1 -- 172.21.15.40:0/1274205621 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7efcb4007070 con 0x7efcbc13e3a0 2023-07-29T14:05:46.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.758+0000 7efcb2ffd640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7efca006bfe0 0x7efca006e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:46.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.759+0000 7efcc246a640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7efca006bfe0 0x7efca006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:46.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.759+0000 7efcc246a640 10 monclient: get_auth_request con 0x7efca006bfe0 auth_method 0 2023-07-29T14:05:46.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.759+0000 7efcc246a640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7efca006bfe0 0x7efca006e490 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efcb80017f0 tx=0x7efcb8009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:46.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.759+0000 7efcb2ffd640 10 client.14904.objecter ms_handle_connect 0x7efca006bfe0 2023-07-29T14:05:46.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.771+0000 7efcb2ffd640 1 -- 172.21.15.40:0/1274205621 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7efcb405c2b0 con 0x7efcbc13e3a0 2023-07-29T14:05:46.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.771+0000 7efcb2ffd640 10 client.14904.objecter ms_dispatch 0x7efcbc000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:46.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.771+0000 7efcb2ffd640 3 client.14904.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:46.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.771+0000 7efcb2ffd640 3 client.14904.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:46.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.771+0000 7efcb2ffd640 20 client.14904.objecter dump_active .. 0 homeless 2023-07-29T14:05:46.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.771+0000 7efcb2ffd640 1 -- 172.21.15.40:0/1274205621 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7efcb40546e0 con 0x7efcbc13e3a0 2023-07-29T14:05:46.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.771+0000 7efcb2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:46.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.771+0000 7efcb2ffd640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:46.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.977+0000 7efcc46ce640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-07-29T14:05:46.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.977+0000 7efcc46ce640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T14:05:46.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.977+0000 7efcc46ce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:46.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:46.977+0000 7efcc46ce640 1 -- 172.21.15.40:0/1274205621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcbc067860 con 0x7efcbc13e3a0 2023-07-29T14:05:47.009 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:42.288 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:47.009 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:42.292 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:47.009 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:42.333 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:47.010 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:42.339 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:47.017 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:42.302 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:47.050 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:42.384 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:47.051 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:42.386 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:47.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.054+0000 7efcb2ffd640 1 -- 172.21.15.40:0/1274205621 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v1004) v1 ==== 74+0+17106 (secure 0 0 0) 0x7efcb4057d00 con 0x7efcbc13e3a0 2023-07-29T14:05:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.054+0000 7efcb2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-07-29T14:05:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.054+0000 7efcb2ffd640 10 monclient: _finish_command 2 = system:0 2023-07-29T14:05:47.069 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T14:05:47.069 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1004,"fsid":"d11937c2-52a1-49ac-aadc-1ae254eda669","created":"2023-07-29T13:27:17.185104+0000","modified":"2023-07-29T14:05:38.273881+0000","last_up_change":"2023-07-29T14:05:36.129005+0000","last_in_change":"2023-07-29T13:27:22.831496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":99,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":259,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-29T13:28:05.611243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-29T13:28:11.546076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"ceph_test_rados_api_asio","create_time":"2023-07-29T13:29:15.052269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6069a944-26b4-4b00-94e2-f91c485e8957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":17,"last_clean_end":1000,"up_from":1002,"up_thru":1002,"down_at":995,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":85570},{"type":"v1","addr":"172.21.15.40:6809","nonce":85570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":1085570},{"type":"v1","addr":"172.21.15.40:6819","nonce":1085570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":85570},{"type":"v1","addr":"172.21.15.40:6815","nonce":85570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":85570},{"type":"v1","addr":"172.21.15.40:6813","nonce":85570}]},"public_addr":"172.21.15.40:6809/85570","cluster_addr":"172.21.15.40:6819/1085570","heartbeat_back_addr":"172.21.15.40:6815/85570","heartbeat_front_addr":"172.21.15.40:6813/85570","state":["exists","up"]},{"osd":1,"uuid":"ab18edd4-e1cb-4815-b5a8-af2b4b2e790b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":17,"last_clean_end":445,"up_from":447,"up_thru":1002,"down_at":443,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":85573},{"type":"v1","addr":"172.21.15.40:6817","nonce":85573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1085573},{"type":"v1","addr":"172.21.15.40:6833","nonce":1085573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":85573},{"type":"v1","addr":"172.21.15.40:6823","nonce":85573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":85573},{"type":"v1","addr":"172.21.15.40:6821","nonce":85573}]},"public_addr":"172.21.15.40:6817/85573","cluster_addr":"172.21.15.40:6833/1085573","heartbeat_back_addr":"172.21.15.40:6823/85573","heartbeat_front_addr":"172.21.15.40:6821/85573","state":["exists","up"]},{"osd":2,"uuid":"4575bc2c-4d43-46d3-95ba-1dc2c94278e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":1002,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":85572},{"type":"v1","addr":"172.21.15.40:6801","nonce":85572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":85572},{"type":"v1","addr":"172.21.15.40:6803","nonce":85572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":85572},{"type":"v1","addr":"172.21.15.40:6807","nonce":85572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":85572},{"type":"v1","addr":"172.21.15.40:6805","nonce":85572}]},"public_addr":"172.21.15.40:6801/85572","cluster_addr":"172.21.15.40:6803/85572","heartbeat_back_addr":"172.21.15.40:6807/85572","heartbeat_front_addr":"172.21.15.40:6805/85572","state":["exists","up"]},{"osd":3,"uuid":"e7224640-7d33-4bb7-81a5-99fc26da7763","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":1002,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":85571},{"type":"v1","addr":"172.21.15.40:6825","nonce":85571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":85571},{"type":"v1","addr":"172.21.15.40:6827","nonce":85571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":85571},{"type":"v1","addr":"172.21.15.40:6831","nonce":85571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":85571},{"type":"v1","addr":"172.21.15.40:6829","nonce":85571}]},"public_addr":"172.21.15.40:6825/85571","cluster_addr":"172.21.15.40:6827/85571","heartbeat_back_addr":"172.21.15.40:6831/85571","heartbeat_front_addr":"172.21.15.40:6829/85571","state":["exists","up"]},{"osd":4,"uuid":"23cd76f6-788b-48cf-9438-e931659d3a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":1002,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":85183},{"type":"v1","addr":"172.21.15.45:6827","nonce":85183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":85183},{"type":"v1","addr":"172.21.15.45:6829","nonce":85183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":85183},{"type":"v1","addr":"172.21.15.45:6833","nonce":85183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":85183},{"type":"v1","addr":"172.21.15.45:6831","nonce":85183}]},"public_addr":"172.21.15.45:6827/85183","cluster_addr":"172.21.15.45:6829/85183","heartbeat_back_addr":"172.21.15.45:6833/85183","heartbeat_front_addr":"172.21.15.45:6831/85183","state":["exists","up"]},{"osd":5,"uuid":"0ddb1e3f-0f65-40d6-ab50-5dc521774445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":702,"up_from":704,"up_thru":1002,"down_at":699,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":85181},{"type":"v1","addr":"172.21.15.45:6813","nonce":85181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3085181},{"type":"v1","addr":"172.21.15.45:6835","nonce":3085181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6823","nonce":85181},{"type":"v1","addr":"172.21.15.45:6825","nonce":85181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":85181},{"type":"v1","addr":"172.21.15.45:6821","nonce":85181}]},"public_addr":"172.21.15.45:6813/85181","cluster_addr":"172.21.15.45:6835/3085181","heartbeat_back_addr":"172.21.15.45:6825/85181","heartbeat_front_addr":"172.21.15.45:6821/85181","state":["exists","up"]},{"osd":6,"uuid":"4b9d2830-9f2c-438c-9c0b-b6692e21f921","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":998,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":85184},{"type":"v1","addr":"172.21.15.45:6811","nonce":85184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":85184},{"type":"v1","addr":"172.21.15.45:6815","nonce":85184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":85184},{"type":"v1","addr":"172.21.15.45:6824","nonce":85184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":85184},{"type":"v1","addr":"172.21.15.45:6819","nonce":85184}]},"public_addr":"172.21.15.45:6811/85184","cluster_addr":"172.21.15.45:6815/85184","heartbeat_back_addr":"172.21.15.45:6824/85184","heartbeat_front_addr":"172.21.15.45:6819/85184","state":["exists","up"]},{"osd":7,"uuid":"6593807d-34a1-431b-aa23-24e8340d887b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":649,"up_from":650,"up_thru":998,"down_at":647,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":85182},{"type":"v1","addr":"172.21.15.45:6803","nonce":85182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2085182},{"type":"v1","addr":"172.21.15.45:6805","nonce":2085182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":85182},{"type":"v1","addr":"172.21.15.45:6809","nonce":85182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":85182},{"type":"v1","addr":"172.21.15.45:6807","nonce":85182}]},"public_addr":"172.21.15.45:6803/85182","cluster_addr":"172.21.15.45:6805/2085182","heartbeat_back_addr":"172.21.15.45:6809/85182","heartbeat_front_addr":"172.21.15.45:6807/85182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2023-07-29T14:05:28.406991+0000","laggy_probability":0.30000001192092896,"laggy_interval":2,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.729512+0000","dead_epoch":999},{"osd":1,"down_stamp":"2023-07-29T13:41:32.068795+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.651149+0000","dead_epoch":444},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:02.683349+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:03.275000+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:03.102406+0000","dead_epoch":0},{"osd":5,"down_stamp":"2023-07-29T13:56:47.268035+0000","laggy_probability":0.65700000524520874,"laggy_interval":7,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:01.537435+0000","dead_epoch":701},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:01.740924+0000","dead_epoch":0},{"osd":7,"down_stamp":"2023-07-29T13:53:13.190217+0000","laggy_probability":0.50999999046325684,"laggy_interval":2,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-29T13:28:00.468651+0000","dead_epoch":648}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":13,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":19,"snaps":[{"begin":9,"length":2}]},{"pool":55,"snaps":[{"begin":11,"length":1}]},{"pool":78,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":86,"snaps":[{"begin":2,"length":1}]},{"pool":93,"snaps":[{"begin":6,"length":2}]},{"pool":101,"snaps":[{"begin":1,"length":1},{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":112,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2},{"begin":10,"length":2}]},{"pool":176,"snaps":[{"begin":12,"length":1}]},{"pool":186,"snaps":[{"begin":14,"length":1}]},{"pool":191,"snaps":[{"begin":17,"length":1}]},{"pool":194,"snaps":[{"begin":19,"length":2}]},{"pool":197,"snaps":[{"begin":24,"length":1}]},{"pool":240,"snaps":[{"begin":2,"length":1}]},{"pool":241,"snaps":[{"begin":4,"length":1}]},{"pool":244,"snaps":[{"begin":6,"length":1}]},{"pool":248,"snaps":[{"begin":9,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-29T14:05:47.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 10 librados: watch_flush enter 2023-07-29T14:05:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 10 librados: watch_flush exit 2023-07-29T14:05:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 20 client.14904.objecter shutdown clearing up homeless session... 2023-07-29T14:05:47.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 10 client.14904.objecter successfully canceled tick 2023-07-29T14:05:47.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 -- 172.21.15.40:0/1274205621 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7efca006bfe0 msgr2=0x7efca006e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:47.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7efca006bfe0 0x7efca006e490 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efcb80017f0 tx=0x7efcb8009040 comp rx=0 tx=0).stop 2023-07-29T14:05:47.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 10 monclient: shutdown 2023-07-29T14:05:47.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:47.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 -- 172.21.15.40:0/1274205621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcbc13e3a0 msgr2=0x7efcbc1384c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:47.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcbc13e3a0 0x7efcbc1384c0 secure :-1 s=READY pgs=3210 cs=0 l=1 rev1=1 crypto rx=0x7efcb400cb30 tx=0x7efcb400a060 comp rx=0 tx=0).stop 2023-07-29T14:05:47.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 -- 172.21.15.40:0/1274205621 shutdown_connections 2023-07-29T14:05:47.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7efca006bfe0 0x7efca006e490 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:47.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcbc13e3a0 0x7efcbc1384c0 unknown :-1 s=CLOSED pgs=3210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:47.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7efcbc0b3ae0 0x7efcbc137f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:47.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 --2- 172.21.15.40:0/1274205621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcbc0a8910 0x7efcbc1d9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:47.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 -- 172.21.15.40:0/1274205621 >> 172.21.15.40:0/1274205621 conn(0x7efcbc070fc0 msgr2=0x7efcbc0b7030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:47.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 -- 172.21.15.40:0/1274205621 shutdown_connections 2023-07-29T14:05:47.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 -- 172.21.15.40:0/1274205621 wait complete. 2023-07-29T14:05:47.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:47.056+0000 7efcc46ce640 1 librados: shutdown 2023-07-29T14:05:47.082 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:42.400 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:47.082 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:42.403 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:48.088 INFO:tasks.ceph:Scrubbing osd.0 2023-07-29T14:05:48.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-07-29T14:05:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.223+0000 7f5b0e383640 10 monclient: get_monmap_and_config 2023-07-29T14:05:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.223+0000 7f5b0e383640 10 monclient: build_initial_monmap 2023-07-29T14:05:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.223+0000 7f5b0e383640 10 monclient: monmap: 2023-07-29T14:05:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:48.223830+0000 2023-07-29T14:05:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:48.223830+0000 2023-07-29T14:05:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:48.229 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.224+0000 7f5b0e383640 1 Processor -- start 2023-07-29T14:05:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.224+0000 7f5b0e383640 1 -- start start 2023-07-29T14:05:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.224+0000 7f5b0e383640 10 monclient: init 2023-07-29T14:05:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2023-07-29T14:05:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b080b3a60 0x7f5b0813de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 10 monclient(hunting): picked mon.noname-b con 0x7f5b080b3a60 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5b0813e380 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5b08141330 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b080b3e30 con 0x7f5b0813e380 2023-07-29T14:05:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b080a8930 con 0x7f5b08141330 2023-07-29T14:05:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b080b0e70 con 0x7f5b080b3a60 2023-07-29T14:05:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b0e383640 10 monclient(hunting): authenticate will time out at 3670.357171s 2023-07-29T14:05:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b077fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5b0813e380 0x7f5b08140c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51766/0 (socket says 172.21.15.40:51766) 2023-07-29T14:05:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.225+0000 7f5b077fe640 1 -- 172.21.15.40:0/803657495 learned_addr learned my addr 172.21.15.40:0/803657495 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:48.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.228+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879997815 0 0) 0x7f5b080b3e30 con 0x7f5b0813e380 2023-07-29T14:05:48.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.228+0000 7f5b06ffd640 10 monclient(hunting): _negotiate 2023-07-29T14:05:48.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.228+0000 7f5b06ffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:48.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.228+0000 7f5b06ffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:48.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.228+0000 7f5b06ffd640 10 monclient(hunting): my global_id is 14907 2023-07-29T14:05:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.228+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af4003520 con 0x7f5b0813e380 2023-07-29T14:05:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4108729453 0 0) 0x7f5af4003520 con 0x7f5b0813e380 2023-07-29T14:05:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b080b3a60 msgr2=0x7f5b0813de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 1 --2- 172.21.15.40:0/803657495 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b080b3a60 0x7f5b0813de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 >> v1:172.21.15.45:6789/0 conn(0x7f5b08141330 legacy=0x7f5b0814b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 10 monclient: _finish_hunting 0 2023-07-29T14:05:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 1 monclient: found mon.noname-a-legacy 2023-07-29T14:05:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.40:6789/0 2023-07-29T14:05:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b080a9600 con 0x7f5b0813e380 2023-07-29T14:05:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 10 monclient: _finish_auth 0 2023-07-29T14:05:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3319386150 0 0) 0x7f5af8001670 con 0x7f5b0813e380 2023-07-29T14:05:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-07-29T14:05:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 10 monclient: dump: 2023-07-29T14:05:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:48.252 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] but i'm connected to v1:172.21.15.40:6789/0, reconnecting 2023-07-29T14:05:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 >> v1:172.21.15.40:6789/0 conn(0x7f5b0813e380 legacy=0x7f5b08140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 10 monclient: _add_conns ranks=[1,0,2] 2023-07-29T14:05:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 1 --2- 172.21.15.40:0/803657495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5af40039a0 0x7f5af4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.231+0000 7f5b06ffd640 10 monclient(hunting): picked mon.b con 0x7f5af40039a0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 1 --2- 172.21.15.40:0/803657495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af40047c0 0x7f5af4004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 10 monclient(hunting): picked mon.a con 0x7f5af40047c0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 1 --2- 172.21.15.40:0/803657495 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b080b3a60 0x7f5af400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 10 monclient(hunting): picked mon.c con 0x7f5b080b3a60 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b080b0e70 con 0x7f5af40047c0 2023-07-29T14:05:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40055b0 con 0x7f5af40039a0 2023-07-29T14:05:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af400bec0 con 0x7f5b080b3a60 2023-07-29T14:05:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.232+0000 7f5b06ffd640 10 monclient(hunting): _finish_auth 0 2023-07-29T14:05:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.234+0000 7f5b077fe640 1 --2- 172.21.15.40:0/803657495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af40047c0 0x7f5af4004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.234+0000 7f5b077fe640 10 monclient(hunting): get_auth_request con 0x7f5af40047c0 auth_method 0 2023-07-29T14:05:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.235+0000 7f5b077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.235+0000 7f5b077fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.235+0000 7f5b077fe640 10 monclient(hunting): _init_auth already have auth, reseting 2023-07-29T14:05:48.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.235+0000 7f5b077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:48.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.235+0000 7f5b077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:48.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.235+0000 7f5b077fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-07-29T14:05:48.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 10 monclient(hunting): handle_auth_done global_id 14907 payload 293 2023-07-29T14:05:48.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 1 -- 172.21.15.40:0/803657495 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b080b3a60 msgr2=0x7f5af400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:48.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 1 --2- 172.21.15.40:0/803657495 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b080b3a60 0x7f5af400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 1 -- 172.21.15.40:0/803657495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5af40039a0 msgr2=0x7f5af4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 1 --2- 172.21.15.40:0/803657495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5af40039a0 0x7f5af4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 10 monclient: _finish_hunting 0 2023-07-29T14:05:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 1 monclient: found mon.a 2023-07-29T14:05:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 1 -- 172.21.15.40:0/803657495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af4003520 con 0x7f5af40047c0 2023-07-29T14:05:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 10 monclient: _finish_auth 0 2023-07-29T14:05:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 10 monclient: _check_auth_tickets 2023-07-29T14:05:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b0e383640 5 monclient: authenticate success, global_id 14907 2023-07-29T14:05:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.237+0000 7f5b0e383640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T14:05:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.240+0000 7f5b077fe640 1 --2- 172.21.15.40:0/803657495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af40047c0 0x7f5af4004c30 secure :-1 s=READY pgs=3212 cs=0 l=1 rev1=1 crypto rx=0x7f5af800c190 tx=0x7f5af40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cab5ecd1b23d22 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af8005ad0 con 0x7f5af40047c0 2023-07-29T14:05:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b06ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b06ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b06ffd640 10 monclient: dump: 2023-07-29T14:05:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:48.281 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af8005c30 con 0x7f5af40047c0 2023-07-29T14:05:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b06ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b06ffd640 1 -- 172.21.15.40:0/803657495 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af802c810 con 0x7f5af40047c0 2023-07-29T14:05:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b0e383640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b0e383640 4 set_mon_vals no callback set 2023-07-29T14:05:48.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b06ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:48.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b06ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:48.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b06ffd640 10 monclient: dump: 2023-07-29T14:05:48.282 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:48.282 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:48.283 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.283 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.283 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:48.283 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.283 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:48.283 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:48.283 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:48.283 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b0e383640 10 monclient: shutdown 2023-07-29T14:05:48.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b0e383640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:48.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b0e383640 1 -- 172.21.15.40:0/803657495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af40047c0 msgr2=0x7f5af4004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b0e383640 1 --2- 172.21.15.40:0/803657495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af40047c0 0x7f5af4004c30 secure :-1 s=READY pgs=3212 cs=0 l=1 rev1=1 crypto rx=0x7f5af800c190 tx=0x7f5af40057f0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b0c920640 1 -- 172.21.15.40:0/803657495 reap_dead start 2023-07-29T14:05:48.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b0e383640 1 -- 172.21.15.40:0/803657495 shutdown_connections 2023-07-29T14:05:48.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b0e383640 1 -- 172.21.15.40:0/803657495 >> 172.21.15.40:0/803657495 conn(0x7f5b08065920 msgr2=0x7f5b08065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:48.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b0e383640 1 -- 172.21.15.40:0/803657495 shutdown_connections 2023-07-29T14:05:48.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.244+0000 7f5b0e383640 1 -- 172.21.15.40:0/803657495 wait complete. 2023-07-29T14:05:48.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.245+0000 7f5b0e383640 10 monclient: build_initial_monmap 2023-07-29T14:05:48.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.245+0000 7f5b0e383640 10 monclient: monmap: 2023-07-29T14:05:48.287 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:48.287 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:48.287 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:48.287 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:48.287 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:48.287 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.287 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:48.287 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:48.288 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:48.288 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.245+0000 7f5b0e383640 1 librados: starting msgr at 2023-07-29T14:05:48.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.245+0000 7f5b0e383640 1 librados: starting objecter 2023-07-29T14:05:48.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 1 Processor -- start 2023-07-29T14:05:48.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 1 -- start start 2023-07-29T14:05:48.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 1 librados: setting wanted keys 2023-07-29T14:05:48.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 1 librados: calling monclient init 2023-07-29T14:05:48.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 10 monclient: init 2023-07-29T14:05:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T14:05:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5af40047c0 0x7f5b08140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 10 monclient(hunting): picked mon.noname-c con 0x7f5af40047c0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b08143f50 0x7f5b08143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 10 monclient(hunting): picked mon.noname-a con 0x7f5b08143f50 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b080afee0 0x7f5b081d56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 10 monclient(hunting): picked mon.noname-b con 0x7f5b080afee0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b08143850 con 0x7f5b08143f50 2023-07-29T14:05:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b08143ac0 con 0x7f5b080afee0 2023-07-29T14:05:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b081d5c10 con 0x7f5af40047c0 2023-07-29T14:05:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.246+0000 7f5b0e383640 10 monclient(hunting): authenticate will time out at 3670.378939s 2023-07-29T14:05:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.247+0000 7f5b077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b08143f50 0x7f5b08143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.247+0000 7f5b077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b08143f50 0x7f5b08143310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53844/0 (socket says 172.21.15.40:53844) 2023-07-29T14:05:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.247+0000 7f5b077fe640 1 -- 172.21.15.40:0/1954846578 learned_addr learned my addr 172.21.15.40:0/1954846578 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.247+0000 7f5b077fe640 10 monclient(hunting): get_auth_request con 0x7f5b08143f50 auth_method 0 2023-07-29T14:05:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.247+0000 7f5b077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.247+0000 7f5b077fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.247+0000 7f5b077fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.248+0000 7f5b077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.248+0000 7f5b077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.248+0000 7f5b077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.251+0000 7f5b077fe640 10 monclient(hunting): handle_auth_done global_id 14910 payload 824 2023-07-29T14:05:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.251+0000 7f5b077fe640 1 -- 172.21.15.40:0/1954846578 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5af40047c0 msgr2=0x7f5b08140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.251+0000 7f5b077fe640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5af40047c0 0x7f5b08140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.251+0000 7f5b077fe640 1 -- 172.21.15.40:0/1954846578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b080afee0 msgr2=0x7f5b081d56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.251+0000 7f5b077fe640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b080afee0 0x7f5b081d56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.251+0000 7f5b077fe640 10 monclient: _finish_hunting 0 2023-07-29T14:05:48.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.251+0000 7f5b077fe640 1 monclient: found mon.noname-a 2023-07-29T14:05:48.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.251+0000 7f5b077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:48.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.251+0000 7f5b077fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T14:05:48.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.251+0000 7f5b077fe640 1 -- 172.21.15.40:0/1954846578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b081d5dd0 con 0x7f5b08143f50 2023-07-29T14:05:48.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.254+0000 7f5b077fe640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b08143f50 0x7f5b08143310 secure :-1 s=READY pgs=3213 cs=0 l=1 rev1=1 crypto rx=0x7f5af800c2b0 tx=0x7f5af8001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:48.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.254+0000 7f5b04ff9640 10 client.?.objecter ms_handle_connect 0x7f5b08143f50 2023-07-29T14:05:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.254+0000 7f5b04ff9640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.258+0000 7f5b04ff9640 1 -- 172.21.15.40:0/1954846578 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af803c040 con 0x7f5b08143f50 2023-07-29T14:05:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.258+0000 7f5b04ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.258+0000 7f5b04ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T14:05:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.258+0000 7f5b04ff9640 10 monclient: dump: 2023-07-29T14:05:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:48.298 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b04ff9640 10 monclient: _finish_auth 0 2023-07-29T14:05:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b04ff9640 10 monclient: _check_auth_tickets 2023-07-29T14:05:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b04ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b04ff9640 1 -- 172.21.15.40:0/1954846578 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af8005e70 con 0x7f5b08143f50 2023-07-29T14:05:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 5 monclient: authenticate success, global_id 14910 2023-07-29T14:05:48.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 10 monclient: _renew_subs 2023-07-29T14:05:48.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:48.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 1 -- 172.21.15.40:0/1954846578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b081360e0 con 0x7f5b08143f50 2023-07-29T14:05:48.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b04ff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:48.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b04ff9640 1 -- 172.21.15.40:0/1954846578 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af8007290 con 0x7f5b08143f50 2023-07-29T14:05:48.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b04ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:48.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b04ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:48.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b04ff9640 10 monclient: dump: 2023-07-29T14:05:48.302 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:48.302 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:48.302 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.304 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.304 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:48.304 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.304 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:48.304 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:48.305 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:48.305 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b067fc640 4 set_mon_vals no callback set 2023-07-29T14:05:48.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 10 client.14910.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:48.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 10 monclient: _renew_subs 2023-07-29T14:05:48.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:48.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 1 -- 172.21.15.40:0/1954846578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b081365c0 con 0x7f5b08143f50 2023-07-29T14:05:48.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 1 librados: init done 2023-07-29T14:05:48.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 10 client.14910.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:48.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 15 client.14910.objecter _session_command_op_assign -1 1 2023-07-29T14:05:48.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 20 client.14910.objecter _get_session osd=-1 returning homeless 2023-07-29T14:05:48.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 10 client.14910.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:48.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:48.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.259+0000 7f5b0e383640 1 -- 172.21.15.40:0/1954846578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b080a9bc0 con 0x7f5b08143f50 2023-07-29T14:05:48.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.281+0000 7f5b04ff9640 1 -- 172.21.15.40:0/1954846578 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f5af8032070 con 0x7f5b08143f50 2023-07-29T14:05:48.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.281+0000 7f5b04ff9640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f5ae806bfb0 0x7f5ae806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.282+0000 7f5b07fff640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f5ae806bfb0 0x7f5ae806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:48.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.282+0000 7f5b07fff640 10 monclient: get_auth_request con 0x7f5ae806bfb0 auth_method 0 2023-07-29T14:05:48.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.282+0000 7f5b04ff9640 10 client.14910.objecter ms_handle_connect 0x7f5ae806bfb0 2023-07-29T14:05:48.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.282+0000 7f5b07fff640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f5ae806bfb0 0x7f5ae806e460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009710 tx=0x7f5af0006d10 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:48.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 1 -- 172.21.15.40:0/1954846578 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f5af8041c90 con 0x7f5b08143f50 2023-07-29T14:05:48.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 10 client.14910.objecter ms_dispatch 0x7f5b08000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 3 client.14910.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 3 client.14910.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 10 client.14910.objecter checking command 1 2023-07-29T14:05:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] conn(0x7f5ae80737f0 0x7f5ae8075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 20 client.14910.objecter _get_session s=0x7f5ae8073120 osd=0 3 2023-07-29T14:05:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 20 client.14910.objecter put_session s=0x7f5ae8073120 osd=0 3 2023-07-29T14:05:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 15 client.14910.objecter _session_command_op_remove -1 1 2023-07-29T14:05:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 20 client.14910.objecter _get_session s=0x7f5ae8073120 osd=0 3 2023-07-29T14:05:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 20 client.14910.objecter get_session s=0x7f5ae8073120 osd=0 3 2023-07-29T14:05:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 15 client.14910.objecter _session_command_op_assign 0 1 2023-07-29T14:05:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 20 client.14910.objecter put_session s=0x7f5ae8073120 osd=0 4 2023-07-29T14:05:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 10 client.14910.objecter _send_command 1 2023-07-29T14:05:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 1 -- 172.21.15.40:0/1954846578 --> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ae8076350 con 0x7f5ae80737f0 2023-07-29T14:05:48.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 20 client.14910.objecter dump_active .. 0 homeless 2023-07-29T14:05:48.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 1 -- 172.21.15.40:0/1954846578 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=1004) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af80764b0 con 0x7f5b08143f50 2023-07-29T14:05:48.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.285+0000 7f5b04ff9640 10 monclient: handle_get_version_reply finishing 1 version 1004 2023-07-29T14:05:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.286+0000 7f5b0c920640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] conn(0x7f5ae80737f0 0x7f5ae8075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.286+0000 7f5b0c920640 10 monclient: get_auth_request con 0x7f5ae80737f0 auth_method 0 2023-07-29T14:05:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.289+0000 7f5b04ff9640 10 client.14910.objecter ms_handle_connect 0x7f5ae80737f0 2023-07-29T14:05:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.289+0000 7f5b0c920640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] conn(0x7f5ae80737f0 0x7f5ae8075c80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.313+0000 7f5b04ff9640 1 -- 172.21.15.40:0/1954846578 <== osd.0 v2:172.21.15.40:6808/85570 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5ae8076350 con 0x7f5ae80737f0 2023-07-29T14:05:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.313+0000 7f5b04ff9640 10 client.14910.objecter ms_dispatch 0x7f5b08000b90 command_reply(tid 1: 0 ) v1 2023-07-29T14:05:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.313+0000 7f5b04ff9640 10 client.14910.objecter _finish_command 1 = system:0 2023-07-29T14:05:48.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.313+0000 7f5b04ff9640 20 client.14910.objecter put_session s=0x7f5ae8073120 osd=0 4 2023-07-29T14:05:48.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.313+0000 7f5b04ff9640 15 client.14910.objecter _session_command_op_remove 0 1 2023-07-29T14:05:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.328+0000 7f5b0e383640 10 client.14910.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-07-29T14:05:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.328+0000 7f5b0e383640 15 client.14910.objecter _session_command_op_assign -1 2 2023-07-29T14:05:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.328+0000 7f5b0e383640 20 client.14910.objecter _get_session s=0x7f5ae8073120 osd=0 3 2023-07-29T14:05:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.328+0000 7f5b0e383640 20 client.14910.objecter put_session s=0x7f5ae8073120 osd=0 3 2023-07-29T14:05:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.328+0000 7f5b0e383640 20 client.14910.objecter _get_session s=0x7f5ae8073120 osd=0 3 2023-07-29T14:05:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.328+0000 7f5b0e383640 15 client.14910.objecter _session_command_op_remove -1 2 2023-07-29T14:05:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.328+0000 7f5b0e383640 20 client.14910.objecter get_session s=0x7f5ae8073120 osd=0 3 2023-07-29T14:05:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.328+0000 7f5b0e383640 15 client.14910.objecter _session_command_op_assign 0 2 2023-07-29T14:05:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.328+0000 7f5b0e383640 20 client.14910.objecter put_session s=0x7f5ae8073120 osd=0 4 2023-07-29T14:05:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.328+0000 7f5b0e383640 10 client.14910.objecter _send_command 2 2023-07-29T14:05:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.328+0000 7f5b0e383640 1 -- 172.21.15.40:0/1954846578 --> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5b080a98e0 con 0x7f5ae80737f0 2023-07-29T14:05:48.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.342+0000 7f5b04ff9640 1 -- 172.21.15.40:0/1954846578 <== osd.0 v2:172.21.15.40:6808/85570 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5b00001780 con 0x7f5ae80737f0 2023-07-29T14:05:48.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b04ff9640 10 client.14910.objecter ms_dispatch 0x7f5b08000b90 command_reply(tid 2: 0 ) v1 2023-07-29T14:05:48.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b04ff9640 10 client.14910.objecter _finish_command 2 = system:0 2023-07-29T14:05:48.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b04ff9640 20 client.14910.objecter put_session s=0x7f5ae8073120 osd=0 4 2023-07-29T14:05:48.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b04ff9640 15 client.14910.objecter _session_command_op_remove 0 2 2023-07-29T14:05:48.345 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-29T14:05:48.345 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-29T14:05:48.345 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-29T14:05:48.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 10 librados: watch_flush enter 2023-07-29T14:05:48.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 10 librados: watch_flush exit 2023-07-29T14:05:48.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 10 client.14910.objecter close_session for osd.0 2023-07-29T14:05:48.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 1 -- 172.21.15.40:0/1954846578 >> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] conn(0x7f5ae80737f0 msgr2=0x7f5ae8075c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.40:6808/85570,v1:172.21.15.40:6809/85570] conn(0x7f5ae80737f0 0x7f5ae8075c80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 20 client.14910.objecter put_session s=0x7f5ae8073120 osd=0 1 2023-07-29T14:05:48.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 20 client.14910.objecter shutdown clearing up homeless session... 2023-07-29T14:05:48.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 10 client.14910.objecter successfully canceled tick 2023-07-29T14:05:48.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 1 -- 172.21.15.40:0/1954846578 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f5ae806bfb0 msgr2=0x7f5ae806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f5ae806bfb0 0x7f5ae806e460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009710 tx=0x7f5af0006d10 comp rx=0 tx=0).stop 2023-07-29T14:05:48.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 10 monclient: shutdown 2023-07-29T14:05:48.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:48.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 1 -- 172.21.15.40:0/1954846578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b08143f50 msgr2=0x7f5b08143310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 1 --2- 172.21.15.40:0/1954846578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b08143f50 0x7f5b08143310 secure :-1 s=READY pgs=3213 cs=0 l=1 rev1=1 crypto rx=0x7f5af800c2b0 tx=0x7f5af8001930 comp rx=0 tx=0).stop 2023-07-29T14:05:48.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0c920640 1 -- 172.21.15.40:0/1954846578 reap_dead start 2023-07-29T14:05:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 1 -- 172.21.15.40:0/1954846578 shutdown_connections 2023-07-29T14:05:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 1 -- 172.21.15.40:0/1954846578 >> 172.21.15.40:0/1954846578 conn(0x7f5b08065920 msgr2=0x7f5b080b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 1 -- 172.21.15.40:0/1954846578 shutdown_connections 2023-07-29T14:05:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 1 -- 172.21.15.40:0/1954846578 wait complete. 2023-07-29T14:05:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.343+0000 7f5b0e383640 1 librados: shutdown 2023-07-29T14:05:48.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-07-29T14:05:48.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.473+0000 7fd4b53f5640 10 monclient: get_monmap_and_config 2023-07-29T14:05:48.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.473+0000 7fd4b53f5640 10 monclient: build_initial_monmap 2023-07-29T14:05:48.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.473+0000 7fd4b53f5640 10 monclient: monmap: 2023-07-29T14:05:48.474 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:48.474 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:48.474 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:48.474122+0000 2023-07-29T14:05:48.475 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:48.474122+0000 2023-07-29T14:05:48.475 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:48.475 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.475 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:48.475 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:48.475 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:48.476 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:48.476 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:48.476 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.474+0000 7fd4b53f5640 1 Processor -- start 2023-07-29T14:05:48.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.474+0000 7fd4b53f5640 1 -- start start 2023-07-29T14:05:48.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.474+0000 7fd4b53f5640 10 monclient: init 2023-07-29T14:05:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2023-07-29T14:05:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 0x7fd4b00aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 10 monclient(hunting): picked mon.noname-b con 0x7fd4b00b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd4b013dde0 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd4b01445e0 0x7fd4b0144a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 10 monclient(hunting): picked mon.noname-c con 0x7fd4b01445e0 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b00b7870 con 0x7fd4b013dde0 2023-07-29T14:05:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b00a91b0 con 0x7fd4b01445e0 2023-07-29T14:05:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00a9490 con 0x7fd4b00b74a0 2023-07-29T14:05:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4b53f5640 10 monclient(hunting): authenticate will time out at 3670.607299s 2023-07-29T14:05:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4aeffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 0x7fd4b00aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4ae7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd4b013dde0 0x7fd4b013e1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51770/0 (socket says 172.21.15.40:51770) 2023-07-29T14:05:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4ae7fc640 1 -- 172.21.15.40:0/54422811 learned_addr learned my addr 172.21.15.40:0/54422811 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.475+0000 7fd4aeffd640 10 monclient(hunting): get_auth_request con 0x7fd4b00b74a0 auth_method 0 2023-07-29T14:05:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.476+0000 7fd4aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.476+0000 7fd4aeffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.476+0000 7fd4aeffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.476+0000 7fd4aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.476+0000 7fd4aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.476+0000 7fd4aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.477+0000 7fd4af7fe640 1 --2- 172.21.15.40:0/54422811 >> v2:172.21.15.45:3300/0 conn(0x7fd4b01445e0 0x7fd4b0144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.477+0000 7fd4af7fe640 10 monclient(hunting): get_auth_request con 0x7fd4b01445e0 auth_method 0 2023-07-29T14:05:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.477+0000 7fd4af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.477+0000 7fd4af7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.477+0000 7fd4af7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4adffb640 1 -- 172.21.15.40:0/54422811 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937070049 0 0) 0x7fd4b00b7870 con 0x7fd4b013dde0 2023-07-29T14:05:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4adffb640 10 monclient(hunting): _negotiate 2023-07-29T14:05:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4adffb640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4adffb640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4adffb640 10 monclient(hunting): my global_id is 14913 2023-07-29T14:05:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4adffb640 1 -- 172.21.15.40:0/54422811 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd498003520 con 0x7fd4b013dde0 2023-07-29T14:05:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4aeffd640 10 monclient(hunting): handle_auth_done global_id 16361 payload 274 2023-07-29T14:05:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4aeffd640 1 -- 172.21.15.40:0/54422811 >> v2:172.21.15.45:3300/0 conn(0x7fd4b01445e0 msgr2=0x7fd4b0144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4aeffd640 1 --2- 172.21.15.40:0/54422811 >> v2:172.21.15.45:3300/0 conn(0x7fd4b01445e0 0x7fd4b0144a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4aeffd640 1 -- 172.21.15.40:0/54422811 >> v1:172.21.15.40:6789/0 conn(0x7fd4b013dde0 legacy=0x7fd4b013e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4aeffd640 10 monclient: _finish_hunting 0 2023-07-29T14:05:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4aeffd640 1 monclient: found mon.noname-b 2023-07-29T14:05:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4aeffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.478+0000 7fd4aeffd640 1 -- 172.21.15.40:0/54422811 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00a9770 con 0x7fd4b00b74a0 2023-07-29T14:05:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.480+0000 7fd4aeffd640 1 --2- 172.21.15.40:0/54422811 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 0x7fd4b00aa150 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009790 tx=0x7fd4a002b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70bebae5b0c732bb server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 1 -- 172.21.15.40:0/54422811 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0031770 con 0x7fd4b00b74a0 2023-07-29T14:05:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 10 monclient: dump: 2023-07-29T14:05:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.497 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:48.497 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.497 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:48.497 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:48.497 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:48.497 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 10 monclient: _finish_auth 0 2023-07-29T14:05:48.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 10 monclient: _check_auth_tickets 2023-07-29T14:05:48.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:48.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 1 -- 172.21.15.40:0/54422811 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a00318d0 con 0x7fd4b00b74a0 2023-07-29T14:05:48.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:48.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 1 -- 172.21.15.40:0/54422811 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0031c60 con 0x7fd4b00b74a0 2023-07-29T14:05:48.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4b53f5640 5 monclient: authenticate success, global_id 16361 2023-07-29T14:05:48.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4b53f5640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:48.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4b53f5640 4 set_mon_vals no callback set 2023-07-29T14:05:48.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:48.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:48.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4adffb640 10 monclient: dump: 2023-07-29T14:05:48.499 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:48.499 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:48.499 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.500 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.500 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:48.500 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.500 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:48.500 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:48.501 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:48.501 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4b53f5640 10 monclient: shutdown 2023-07-29T14:05:48.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4b53f5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4b53f5640 1 -- 172.21.15.40:0/54422811 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 msgr2=0x7fd4b00aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4b53f5640 1 --2- 172.21.15.40:0/54422811 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 0x7fd4b00aa150 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009790 tx=0x7fd4a002b810 comp rx=0 tx=0).stop 2023-07-29T14:05:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4b53f5640 1 -- 172.21.15.40:0/54422811 shutdown_connections 2023-07-29T14:05:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4b53f5640 1 --2- 172.21.15.40:0/54422811 >> v2:172.21.15.45:3300/0 conn(0x7fd4b01445e0 0x7fd4b0144a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4b53f5640 1 --2- 172.21.15.40:0/54422811 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 0x7fd4b00aa150 unknown :-1 s=CLOSED pgs=3219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.484+0000 7fd4b53f5640 1 -- 172.21.15.40:0/54422811 >> 172.21.15.40:0/54422811 conn(0x7fd4b0065b00 msgr2=0x7fd4b0065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.485+0000 7fd4b53f5640 1 -- 172.21.15.40:0/54422811 shutdown_connections 2023-07-29T14:05:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.485+0000 7fd4b53f5640 1 -- 172.21.15.40:0/54422811 wait complete. 2023-07-29T14:05:48.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.485+0000 7fd4b53f5640 10 monclient: build_initial_monmap 2023-07-29T14:05:48.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.485+0000 7fd4b53f5640 10 monclient: monmap: 2023-07-29T14:05:48.503 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:48.503 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:48.503 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:48.503 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:48.503 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:48.504 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.504 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:48.504 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:48.505 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:48.505 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.485+0000 7fd4b53f5640 1 librados: starting msgr at 2023-07-29T14:05:48.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.485+0000 7fd4b53f5640 1 librados: starting objecter 2023-07-29T14:05:48.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 1 Processor -- start 2023-07-29T14:05:48.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 1 -- start start 2023-07-29T14:05:48.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 1 librados: setting wanted keys 2023-07-29T14:05:48.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 1 librados: calling monclient init 2023-07-29T14:05:48.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 10 monclient: init 2023-07-29T14:05:48.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:48.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T14:05:48.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 0x7fd4b01438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 10 monclient(hunting): picked mon.noname-c con 0x7fd4b00b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4b013dde0 0x7fd4b00b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 10 monclient(hunting): picked mon.noname-a con 0x7fd4b013dde0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4b01445e0 0x7fd4b00ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 10 monclient(hunting): picked mon.noname-b con 0x7fd4b01445e0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00b01b0 con 0x7fd4b013dde0 2023-07-29T14:05:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00b10d0 con 0x7fd4b01445e0 2023-07-29T14:05:48.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:48.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00b0670 con 0x7fd4b00b74a0 2023-07-29T14:05:48.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:48.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.486+0000 7fd4b53f5640 10 monclient(hunting): authenticate will time out at 3670.618909s 2023-07-29T14:05:48.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4aeffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 0x7fd4b01438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:48.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4ae7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4b013dde0 0x7fd4b00b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:48.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4af7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4b01445e0 0x7fd4b00ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:48.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4aeffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 0x7fd4b01438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49500/0 (socket says 172.21.15.40:49500) 2023-07-29T14:05:48.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4aeffd640 1 -- 172.21.15.40:0/277462954 learned_addr learned my addr 172.21.15.40:0/277462954 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:48.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4aeffd640 10 monclient(hunting): get_auth_request con 0x7fd4b00b74a0 auth_method 0 2023-07-29T14:05:48.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:48.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4aeffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:48.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4aeffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:48.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4af7fe640 10 monclient(hunting): get_auth_request con 0x7fd4b01445e0 auth_method 0 2023-07-29T14:05:48.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:48.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4af7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:48.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4af7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:48.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4ae7fc640 10 monclient(hunting): get_auth_request con 0x7fd4b013dde0 auth_method 0 2023-07-29T14:05:48.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:48.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4ae7fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:48.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.487+0000 7fd4ae7fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.488+0000 7fd4aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.488+0000 7fd4aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.488+0000 7fd4aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.488+0000 7fd4af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.488+0000 7fd4af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.488+0000 7fd4af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.488+0000 7fd4ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.488+0000 7fd4ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.488+0000 7fd4ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.490+0000 7fd4aeffd640 10 monclient(hunting): handle_auth_done global_id 16364 payload 824 2023-07-29T14:05:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.491+0000 7fd4aeffd640 1 -- 172.21.15.40:0/277462954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4b01445e0 msgr2=0x7fd4b00ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.491+0000 7fd4aeffd640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4b01445e0 0x7fd4b00ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.491+0000 7fd4aeffd640 1 -- 172.21.15.40:0/277462954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4b013dde0 msgr2=0x7fd4b00b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.491+0000 7fd4aeffd640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4b013dde0 0x7fd4b00b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.491+0000 7fd4aeffd640 10 monclient: _finish_hunting 0 2023-07-29T14:05:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.491+0000 7fd4aeffd640 1 monclient: found mon.noname-c 2023-07-29T14:05:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.491+0000 7fd4aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.491+0000 7fd4aeffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:48.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.491+0000 7fd4aeffd640 1 -- 172.21.15.40:0/277462954 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a00093a0 con 0x7fd4b00b74a0 2023-07-29T14:05:48.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.493+0000 7fd4aeffd640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 0x7fd4b01438a0 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:48.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.493+0000 7fd4977fe640 10 client.?.objecter ms_handle_connect 0x7fd4b00b74a0 2023-07-29T14:05:48.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.493+0000 7fd4977fe640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 1 -- 172.21.15.40:0/277462954 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0032610 con 0x7fd4b00b74a0 2023-07-29T14:05:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 10 monclient: dump: 2023-07-29T14:05:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:48.534 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:48.534 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.534 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.534 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:48.534 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.534 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:48.534 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:48.534 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:48.535 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 10 monclient: _finish_auth 0 2023-07-29T14:05:48.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 10 monclient: _check_auth_tickets 2023-07-29T14:05:48.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:48.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 1 -- 172.21.15.40:0/277462954 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a0032770 con 0x7fd4b00b74a0 2023-07-29T14:05:48.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:48.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 1 -- 172.21.15.40:0/277462954 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0032b00 con 0x7fd4b00b74a0 2023-07-29T14:05:48.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4b53f5640 5 monclient: authenticate success, global_id 16364 2023-07-29T14:05:48.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4b53f5640 10 monclient: _renew_subs 2023-07-29T14:05:48.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4b53f5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:48.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4b53f5640 1 -- 172.21.15.40:0/277462954 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b00ae6e0 con 0x7fd4b00b74a0 2023-07-29T14:05:48.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:48.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:48.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4977fe640 10 monclient: dump: 2023-07-29T14:05:48.536 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:48.537 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:48.537 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.537 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:48.537 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:48.537 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:48.537 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:48.537 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:48.543 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:48.543 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:48.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4ad7fa640 4 set_mon_vals no callback set 2023-07-29T14:05:48.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4b53f5640 10 client.16364.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:48.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4b53f5640 10 monclient: _renew_subs 2023-07-29T14:05:48.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4b53f5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:48.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4b53f5640 1 -- 172.21.15.40:0/277462954 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b00b3ee0 con 0x7fd4b00b74a0 2023-07-29T14:05:48.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.497+0000 7fd4b53f5640 1 librados: init done 2023-07-29T14:05:48.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.498+0000 7fd4b53f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:48.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.498+0000 7fd4b53f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:48.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.498+0000 7fd4b53f5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:48.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.498+0000 7fd4b53f5640 1 -- 172.21.15.40:0/277462954 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b004e000 con 0x7fd4b00b74a0 2023-07-29T14:05:48.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.517+0000 7fd4977fe640 1 -- 172.21.15.40:0/277462954 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fd4a0032c60 con 0x7fd4b00b74a0 2023-07-29T14:05:48.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.518+0000 7fd4977fe640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd49c06bfb0 0x7fd49c06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:48.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.518+0000 7fd4ae7fc640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd49c06bfb0 0x7fd49c06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:48.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.518+0000 7fd4ae7fc640 10 monclient: get_auth_request con 0x7fd49c06bfb0 auth_method 0 2023-07-29T14:05:48.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.518+0000 7fd4ae7fc640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd49c06bfb0 0x7fd49c06e460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0066340 tx=0x7fd4a4007d30 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:48.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.518+0000 7fd4977fe640 10 client.16364.objecter ms_handle_connect 0x7fd49c06bfb0 2023-07-29T14:05:48.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.774+0000 7fd4977fe640 1 -- 172.21.15.40:0/277462954 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7fd4a007dbd0 con 0x7fd4b00b74a0 2023-07-29T14:05:48.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.774+0000 7fd4977fe640 10 client.16364.objecter ms_dispatch 0x7fd4b0000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:48.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.774+0000 7fd4977fe640 3 client.16364.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:48.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.774+0000 7fd4977fe640 3 client.16364.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:48.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.774+0000 7fd4977fe640 20 client.16364.objecter dump_active .. 0 homeless 2023-07-29T14:05:48.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.774+0000 7fd4977fe640 1 -- 172.21.15.40:0/277462954 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7fd4a0076000 con 0x7fd4b00b74a0 2023-07-29T14:05:48.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.774+0000 7fd4977fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:48.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.774+0000 7fd4977fe640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:48.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.951+0000 7fd4b53f5640 1 -- 172.21.15.40:0/277462954 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4b00af140 con 0x7fd49c06bfb0 2023-07-29T14:05:48.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.952+0000 7fd4977fe640 1 -- 172.21.15.40:0/277462954 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd4a4005850 con 0x7fd49c06bfb0 2023-07-29T14:05:48.953 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2023-07-29T14:05:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 10 librados: watch_flush enter 2023-07-29T14:05:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 10 librados: watch_flush exit 2023-07-29T14:05:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 20 client.16364.objecter shutdown clearing up homeless session... 2023-07-29T14:05:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 10 client.16364.objecter successfully canceled tick 2023-07-29T14:05:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 -- 172.21.15.40:0/277462954 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd49c06bfb0 msgr2=0x7fd49c06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd49c06bfb0 0x7fd49c06e460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0066340 tx=0x7fd4a4007d30 comp rx=0 tx=0).stop 2023-07-29T14:05:48.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 10 monclient: shutdown 2023-07-29T14:05:48.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:48.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 -- 172.21.15.40:0/277462954 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 msgr2=0x7fd4b01438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:48.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 0x7fd4b01438a0 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a00046c0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 -- 172.21.15.40:0/277462954 shutdown_connections 2023-07-29T14:05:48.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fd49c06bfb0 0x7fd49c06e460 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4b01445e0 0x7fd4b00ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4b013dde0 0x7fd4b00b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 --2- 172.21.15.40:0/277462954 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd4b00b74a0 0x7fd4b01438a0 unknown :-1 s=CLOSED pgs=3220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:48.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 -- 172.21.15.40:0/277462954 >> 172.21.15.40:0/277462954 conn(0x7fd4b0065b00 msgr2=0x7fd4b00b6420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 -- 172.21.15.40:0/277462954 shutdown_connections 2023-07-29T14:05:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 -- 172.21.15.40:0/277462954 wait complete. 2023-07-29T14:05:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:48.954+0000 7fd4b53f5640 1 librados: shutdown 2023-07-29T14:05:48.965 INFO:tasks.ceph:Scrubbing osd.1 2023-07-29T14:05:48.966 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-07-29T14:05:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.077+0000 7f919dd9b640 10 monclient: get_monmap_and_config 2023-07-29T14:05:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.077+0000 7f919dd9b640 10 monclient: build_initial_monmap 2023-07-29T14:05:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.077+0000 7f919dd9b640 10 monclient: monmap: 2023-07-29T14:05:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:49.077694+0000 2023-07-29T14:05:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:49.077694+0000 2023-07-29T14:05:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:49.082 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:49.082 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:49.082 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 1 Processor -- start 2023-07-29T14:05:49.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 1 -- start start 2023-07-29T14:05:49.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 10 monclient: init 2023-07-29T14:05:49.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2023-07-29T14:05:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f91980b3550 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3920 0x7f91980af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 10 monclient(hunting): picked mon.noname-b con 0x7f91980b3920 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f91980b0170 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9198141e50 con 0x7f91980b0170 2023-07-29T14:05:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91980afd90 con 0x7f91980b3550 2023-07-29T14:05:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919813ec30 con 0x7f91980b3920 2023-07-29T14:05:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.078+0000 7f919dd9b640 10 monclient(hunting): authenticate will time out at 3671.211006s 2023-07-29T14:05:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.079+0000 7f9196ffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3920 0x7f91980af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.079+0000 7f9197fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f91980b0170 0x7f919814b770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51774/0 (socket says 172.21.15.40:51774) 2023-07-29T14:05:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.079+0000 7f9197fff640 1 -- 172.21.15.40:0/1296510295 learned_addr learned my addr 172.21.15.40:0/1296510295 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.079+0000 7f9196ffd640 10 monclient(hunting): get_auth_request con 0x7f91980b3920 auth_method 0 2023-07-29T14:05:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.079+0000 7f9196ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.079+0000 7f9196ffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.079+0000 7f9196ffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.080+0000 7f9196ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.080+0000 7f9196ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.080+0000 7f9196ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.082+0000 7f9196ffd640 10 monclient(hunting): handle_auth_done global_id 16367 payload 274 2023-07-29T14:05:49.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.082+0000 7f91967fc640 1 -- 172.21.15.40:0/1296510295 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947616294 0 0) 0x7f9198141e50 con 0x7f91980b0170 2023-07-29T14:05:49.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.082+0000 7f9196ffd640 1 -- 172.21.15.40:0/1296510295 >> v1:172.21.15.45:6789/0 conn(0x7f91980b3550 legacy=0x7f9198141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.082+0000 7f9196ffd640 1 -- 172.21.15.40:0/1296510295 >> v1:172.21.15.40:6789/0 conn(0x7f91980b0170 legacy=0x7f919814b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.082+0000 7f9196ffd640 10 monclient: _finish_hunting 0 2023-07-29T14:05:49.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.082+0000 7f9196ffd640 1 monclient: found mon.noname-b 2023-07-29T14:05:49.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.082+0000 7f9196ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:49.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.082+0000 7f9196ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.082+0000 7f9196ffd640 1 -- 172.21.15.40:0/1296510295 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91980aeee0 con 0x7f91980b3920 2023-07-29T14:05:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.082+0000 7f91967fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2023-07-29T14:05:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.084+0000 7f9196ffd640 1 --2- 172.21.15.40:0/1296510295 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3920 0x7f91980af850 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 crypto rx=0x7f918c0097d0 tx=0x7f918c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=573a3133fe023bc server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 1 -- 172.21.15.40:0/1296510295 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c031800 con 0x7f91980b3920 2023-07-29T14:05:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 10 monclient: dump: 2023-07-29T14:05:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.108 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 10 monclient: _finish_auth 0 2023-07-29T14:05:49.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 10 monclient: _check_auth_tickets 2023-07-29T14:05:49.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:49.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f919dd9b640 5 monclient: authenticate success, global_id 16367 2023-07-29T14:05:49.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 1 -- 172.21.15.40:0/1296510295 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c031960 con 0x7f91980b3920 2023-07-29T14:05:49.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f919dd9b640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T14:05:49.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:49.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f919dd9b640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:49.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f919dd9b640 4 set_mon_vals no callback set 2023-07-29T14:05:49.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 1 -- 172.21.15.40:0/1296510295 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c031cf0 con 0x7f91980b3920 2023-07-29T14:05:49.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:49.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.088+0000 7f91967fc640 10 monclient: dump: 2023-07-29T14:05:49.110 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.110 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.110 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.110 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.110 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.110 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.110 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.111 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.111 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.111 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.089+0000 7f919dd9b640 10 monclient: shutdown 2023-07-29T14:05:49.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.089+0000 7f919dd9b640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:49.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.089+0000 7f919dd9b640 1 -- 172.21.15.40:0/1296510295 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3920 msgr2=0x7f91980af850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.089+0000 7f919dd9b640 1 --2- 172.21.15.40:0/1296510295 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3920 0x7f91980af850 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 crypto rx=0x7f918c0097d0 tx=0x7f918c02b8a0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.089+0000 7f919dd9b640 1 -- 172.21.15.40:0/1296510295 shutdown_connections 2023-07-29T14:05:49.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.089+0000 7f919dd9b640 1 --2- 172.21.15.40:0/1296510295 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3920 0x7f91980af850 unknown :-1 s=CLOSED pgs=3221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.089+0000 7f919dd9b640 1 -- 172.21.15.40:0/1296510295 >> 172.21.15.40:0/1296510295 conn(0x7f9198065920 msgr2=0x7f9198065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:49.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.089+0000 7f919dd9b640 1 -- 172.21.15.40:0/1296510295 shutdown_connections 2023-07-29T14:05:49.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.089+0000 7f919dd9b640 1 -- 172.21.15.40:0/1296510295 wait complete. 2023-07-29T14:05:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.089+0000 7f919dd9b640 10 monclient: build_initial_monmap 2023-07-29T14:05:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.089+0000 7f919dd9b640 10 monclient: monmap: 2023-07-29T14:05:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:49.116 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:49.116 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.116 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:49.116 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:49.116 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:49.117 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.090+0000 7f919dd9b640 1 librados: starting msgr at 2023-07-29T14:05:49.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.090+0000 7f919dd9b640 1 librados: starting objecter 2023-07-29T14:05:49.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.090+0000 7f919dd9b640 1 Processor -- start 2023-07-29T14:05:49.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.090+0000 7f919dd9b640 1 -- start start 2023-07-29T14:05:49.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.090+0000 7f919dd9b640 1 librados: setting wanted keys 2023-07-29T14:05:49.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.090+0000 7f919dd9b640 1 librados: calling monclient init 2023-07-29T14:05:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.090+0000 7f919dd9b640 10 monclient: init 2023-07-29T14:05:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T14:05:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91980b0170 0x7f91980acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 10 monclient(hunting): picked mon.noname-b con 0x7f91980b0170 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3550 0x7f91980abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 10 monclient(hunting): picked mon.noname-c con 0x7f91980b3550 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91980b3920 0x7f91980ac210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 10 monclient(hunting): picked mon.noname-a con 0x7f91980b3920 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:49.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91980ac750 con 0x7f91980b3920 2023-07-29T14:05:49.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91981d9110 con 0x7f91980b0170 2023-07-29T14:05:49.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91981d92d0 con 0x7f91980b3550 2023-07-29T14:05:49.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f919dd9b640 10 monclient(hunting): authenticate will time out at 3671.223272s 2023-07-29T14:05:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f9196ffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3550 0x7f91980abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f9197fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91980b3920 0x7f91980ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f91977fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91980b0170 0x7f91980acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f9196ffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3550 0x7f91980abcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49520/0 (socket says 172.21.15.40:49520) 2023-07-29T14:05:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f9196ffd640 1 -- 172.21.15.40:0/3589394220 learned_addr learned my addr 172.21.15.40:0/3589394220 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f9196ffd640 10 monclient(hunting): get_auth_request con 0x7f91980b3550 auth_method 0 2023-07-29T14:05:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f9196ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f9196ffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.091+0000 7f9196ffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f91977fe640 10 monclient(hunting): get_auth_request con 0x7f91980b0170 auth_method 0 2023-07-29T14:05:49.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f91977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f91977fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f91977fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f9197fff640 10 monclient(hunting): get_auth_request con 0x7f91980b3920 auth_method 0 2023-07-29T14:05:49.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f9197fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f9197fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f9197fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f9196ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f9196ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f9196ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f91977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f91977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f91977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f9197fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f9197fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.092+0000 7f9197fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.095+0000 7f9196ffd640 10 monclient(hunting): handle_auth_done global_id 16370 payload 824 2023-07-29T14:05:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.095+0000 7f9196ffd640 1 -- 172.21.15.40:0/3589394220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91980b0170 msgr2=0x7f91980acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.095+0000 7f9196ffd640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91980b0170 0x7f91980acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.095+0000 7f9196ffd640 1 -- 172.21.15.40:0/3589394220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91980b3920 msgr2=0x7f91980ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.095+0000 7f9196ffd640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91980b3920 0x7f91980ac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.095+0000 7f9196ffd640 10 monclient: _finish_hunting 0 2023-07-29T14:05:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.095+0000 7f9196ffd640 1 monclient: found mon.noname-c 2023-07-29T14:05:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.095+0000 7f9196ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.095+0000 7f9196ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.095+0000 7f9196ffd640 1 -- 172.21.15.40:0/3589394220 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c009400 con 0x7f91980b3550 2023-07-29T14:05:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.097+0000 7f9196ffd640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3550 0x7f91980abcd0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 crypto rx=0x7f918c0097d0 tx=0x7f918c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.097+0000 7f917ffff640 10 client.?.objecter ms_handle_connect 0x7f91980b3550 2023-07-29T14:05:49.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.097+0000 7f917ffff640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:49.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 1 -- 172.21.15.40:0/3589394220 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c0443f0 con 0x7f91980b3550 2023-07-29T14:05:49.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:49.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 10 monclient: dump: 2023-07-29T14:05:49.142 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.142 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.142 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.143 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 10 monclient: _finish_auth 0 2023-07-29T14:05:49.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 10 monclient: _check_auth_tickets 2023-07-29T14:05:49.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 1 -- 172.21.15.40:0/3589394220 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c044570 con 0x7f91980b3550 2023-07-29T14:05:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f919dd9b640 5 monclient: authenticate success, global_id 16370 2023-07-29T14:05:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f919dd9b640 10 monclient: _renew_subs 2023-07-29T14:05:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f919dd9b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f919dd9b640 1 -- 172.21.15.40:0/3589394220 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91981d9490 con 0x7f91980b3550 2023-07-29T14:05:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 1 -- 172.21.15.40:0/3589394220 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c044920 con 0x7f91980b3550 2023-07-29T14:05:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f917ffff640 10 monclient: dump: 2023-07-29T14:05:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.153 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.153 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.153 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.153 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.153 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.153 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.153 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f919dd9b640 10 client.16370.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:49.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f919dd9b640 10 monclient: _renew_subs 2023-07-29T14:05:49.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f919dd9b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f919dd9b640 1 -- 172.21.15.40:0/3589394220 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91981d99a0 con 0x7f91980b3550 2023-07-29T14:05:49.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f919dd9b640 1 librados: init done 2023-07-29T14:05:49.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.101+0000 7f9195ffb640 4 set_mon_vals no callback set 2023-07-29T14:05:49.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.102+0000 7f919dd9b640 10 client.16370.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:49.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.102+0000 7f919dd9b640 15 client.16370.objecter _session_command_op_assign -1 1 2023-07-29T14:05:49.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.102+0000 7f919dd9b640 20 client.16370.objecter _get_session osd=-1 returning homeless 2023-07-29T14:05:49.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.102+0000 7f919dd9b640 10 client.16370.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:49.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.102+0000 7f919dd9b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.102+0000 7f919dd9b640 1 -- 172.21.15.40:0/3589394220 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9198141220 con 0x7f91980b3550 2023-07-29T14:05:49.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.123+0000 7f917ffff640 1 -- 172.21.15.40:0/3589394220 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f918c04d450 con 0x7f91980b3550 2023-07-29T14:05:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.124+0000 7f917ffff640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f918406bfb0 0x7f918406e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.124+0000 7f91977fe640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f918406bfb0 0x7f918406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.124+0000 7f91977fe640 10 monclient: get_auth_request con 0x7f918406bfb0 auth_method 0 2023-07-29T14:05:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.124+0000 7f917ffff640 10 client.16370.objecter ms_handle_connect 0x7f918406bfb0 2023-07-29T14:05:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.124+0000 7f91977fe640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f918406bfb0 0x7f918406e460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9188000f60 tx=0x7f9188009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 1 -- 172.21.15.40:0/3589394220 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f918c02e070 con 0x7f91980b3550 2023-07-29T14:05:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 10 client.16370.objecter ms_dispatch 0x7f9198000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 3 client.16370.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 3 client.16370.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 10 client.16370.objecter checking command 1 2023-07-29T14:05:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] conn(0x7f91840737f0 0x7f9184075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 20 client.16370.objecter _get_session s=0x7f9184073120 osd=1 3 2023-07-29T14:05:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 20 client.16370.objecter put_session s=0x7f9184073120 osd=1 3 2023-07-29T14:05:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 15 client.16370.objecter _session_command_op_remove -1 1 2023-07-29T14:05:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 20 client.16370.objecter _get_session s=0x7f9184073120 osd=1 3 2023-07-29T14:05:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 20 client.16370.objecter get_session s=0x7f9184073120 osd=1 3 2023-07-29T14:05:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 15 client.16370.objecter _session_command_op_assign 1 1 2023-07-29T14:05:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 20 client.16370.objecter put_session s=0x7f9184073120 osd=1 4 2023-07-29T14:05:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 10 client.16370.objecter _send_command 1 2023-07-29T14:05:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 1 -- 172.21.15.40:0/3589394220 --> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9184076350 con 0x7f91840737f0 2023-07-29T14:05:49.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f917ffff640 20 client.16370.objecter dump_active .. 0 homeless 2023-07-29T14:05:49.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.126+0000 7f9197fff640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] conn(0x7f91840737f0 0x7f9184075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.127+0000 7f9197fff640 10 monclient: get_auth_request con 0x7f91840737f0 auth_method 0 2023-07-29T14:05:49.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.127+0000 7f917ffff640 1 -- 172.21.15.40:0/3589394220 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=1004) v2 ==== 24+0+0 (secure 0 0 0) 0x7f918c076680 con 0x7f91980b3550 2023-07-29T14:05:49.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.127+0000 7f917ffff640 10 monclient: handle_get_version_reply finishing 1 version 1004 2023-07-29T14:05:49.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.130+0000 7f917ffff640 10 client.16370.objecter ms_handle_connect 0x7f91840737f0 2023-07-29T14:05:49.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.130+0000 7f9197fff640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] conn(0x7f91840737f0 0x7f9184075c80 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.153+0000 7f917ffff640 1 -- 172.21.15.40:0/3589394220 <== osd.1 v2:172.21.15.40:6816/85573 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9184076350 con 0x7f91840737f0 2023-07-29T14:05:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.153+0000 7f917ffff640 10 client.16370.objecter ms_dispatch 0x7f9198000b90 command_reply(tid 1: 0 ) v1 2023-07-29T14:05:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.153+0000 7f917ffff640 10 client.16370.objecter _finish_command 1 = system:0 2023-07-29T14:05:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.153+0000 7f917ffff640 20 client.16370.objecter put_session s=0x7f9184073120 osd=1 4 2023-07-29T14:05:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.153+0000 7f917ffff640 15 client.16370.objecter _session_command_op_remove 1 1 2023-07-29T14:05:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.167+0000 7f919dd9b640 10 client.16370.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-07-29T14:05:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.167+0000 7f919dd9b640 15 client.16370.objecter _session_command_op_assign -1 2 2023-07-29T14:05:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.167+0000 7f919dd9b640 20 client.16370.objecter _get_session s=0x7f9184073120 osd=1 3 2023-07-29T14:05:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.167+0000 7f919dd9b640 20 client.16370.objecter put_session s=0x7f9184073120 osd=1 3 2023-07-29T14:05:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.167+0000 7f919dd9b640 20 client.16370.objecter _get_session s=0x7f9184073120 osd=1 3 2023-07-29T14:05:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.167+0000 7f919dd9b640 15 client.16370.objecter _session_command_op_remove -1 2 2023-07-29T14:05:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.167+0000 7f919dd9b640 20 client.16370.objecter get_session s=0x7f9184073120 osd=1 3 2023-07-29T14:05:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.167+0000 7f919dd9b640 15 client.16370.objecter _session_command_op_assign 1 2 2023-07-29T14:05:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.167+0000 7f919dd9b640 20 client.16370.objecter put_session s=0x7f9184073120 osd=1 4 2023-07-29T14:05:49.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.167+0000 7f919dd9b640 10 client.16370.objecter _send_command 2 2023-07-29T14:05:49.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.167+0000 7f919dd9b640 1 -- 172.21.15.40:0/3589394220 --> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f91980b3d30 con 0x7f91840737f0 2023-07-29T14:05:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f917ffff640 1 -- 172.21.15.40:0/3589394220 <== osd.1 v2:172.21.15.40:6816/85573 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9190001bd0 con 0x7f91840737f0 2023-07-29T14:05:49.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f917ffff640 10 client.16370.objecter ms_dispatch 0x7f9198000b90 command_reply(tid 2: 0 ) v1 2023-07-29T14:05:49.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f917ffff640 10 client.16370.objecter _finish_command 2 = system:0 2023-07-29T14:05:49.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f917ffff640 20 client.16370.objecter put_session s=0x7f9184073120 osd=1 4 2023-07-29T14:05:49.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f917ffff640 15 client.16370.objecter _session_command_op_remove 1 2 2023-07-29T14:05:49.189 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-29T14:05:49.189 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-29T14:05:49.190 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-29T14:05:49.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 10 librados: watch_flush enter 2023-07-29T14:05:49.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 10 librados: watch_flush exit 2023-07-29T14:05:49.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 10 client.16370.objecter close_session for osd.1 2023-07-29T14:05:49.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 1 -- 172.21.15.40:0/3589394220 >> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] conn(0x7f91840737f0 msgr2=0x7f9184075c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.40:6816/85573,v1:172.21.15.40:6817/85573] conn(0x7f91840737f0 0x7f9184075c80 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 20 client.16370.objecter put_session s=0x7f9184073120 osd=1 1 2023-07-29T14:05:49.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 20 client.16370.objecter shutdown clearing up homeless session... 2023-07-29T14:05:49.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 10 client.16370.objecter successfully canceled tick 2023-07-29T14:05:49.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 1 -- 172.21.15.40:0/3589394220 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f918406bfb0 msgr2=0x7f918406e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f918406bfb0 0x7f918406e460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9188000f60 tx=0x7f9188009040 comp rx=0 tx=0).stop 2023-07-29T14:05:49.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 10 monclient: shutdown 2023-07-29T14:05:49.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:49.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 1 -- 172.21.15.40:0/3589394220 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3550 msgr2=0x7f91980abcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 1 --2- 172.21.15.40:0/3589394220 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f91980b3550 0x7f91980abcd0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 crypto rx=0x7f918c0097d0 tx=0x7f918c0046c0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f9197fff640 1 -- 172.21.15.40:0/3589394220 reap_dead start 2023-07-29T14:05:49.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 1 -- 172.21.15.40:0/3589394220 shutdown_connections 2023-07-29T14:05:49.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 1 -- 172.21.15.40:0/3589394220 >> 172.21.15.40:0/3589394220 conn(0x7f9198065920 msgr2=0x7f91980b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:49.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.182+0000 7f919dd9b640 1 -- 172.21.15.40:0/3589394220 shutdown_connections 2023-07-29T14:05:49.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.183+0000 7f919dd9b640 1 -- 172.21.15.40:0/3589394220 wait complete. 2023-07-29T14:05:49.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.183+0000 7f919dd9b640 1 librados: shutdown 2023-07-29T14:05:49.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-07-29T14:05:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.308+0000 7f9d7e674640 10 monclient: get_monmap_and_config 2023-07-29T14:05:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.308+0000 7f9d7e674640 10 monclient: build_initial_monmap 2023-07-29T14:05:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.308+0000 7f9d7e674640 10 monclient: monmap: 2023-07-29T14:05:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:49.309151+0000 2023-07-29T14:05:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:49.309151+0000 2023-07-29T14:05:49.312 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:49.312 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.312 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:49.312 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:49.312 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:49.313 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:49.313 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:49.313 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.309+0000 7f9d7e674640 1 Processor -- start 2023-07-29T14:05:49.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.309+0000 7f9d7e674640 1 -- start start 2023-07-29T14:05:49.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.309+0000 7f9d7e674640 10 monclient: init 2023-07-29T14:05:49.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:49.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2023-07-29T14:05:49.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9d780a86c0 0x7f9d78140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 10 monclient(hunting): picked mon.noname-a con 0x7f9d780a86c0 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:49.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9d781410e0 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:49.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d7813e2e0 0x7f9d7814b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 10 monclient(hunting): picked mon.noname-b con 0x7f9d7813e2e0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:49.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d780a8a90 con 0x7f9d781410e0 2023-07-29T14:05:49.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d780a98e0 con 0x7f9d780a86c0 2023-07-29T14:05:49.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780a9bc0 con 0x7f9d7813e2e0 2023-07-29T14:05:49.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:49.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7e674640 10 monclient(hunting): authenticate will time out at 3671.442414s 2023-07-29T14:05:49.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d7cc11640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d7813e2e0 0x7f9d7814b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.310+0000 7f9d77fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9d780a86c0 0x7f9d78140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d7cc11640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d7813e2e0 0x7f9d7814b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49524/0 (socket says 172.21.15.40:49524) 2023-07-29T14:05:49.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d7cc11640 1 -- 172.21.15.40:0/750480108 learned_addr learned my addr 172.21.15.40:0/750480108 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:49.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d7cc11640 10 monclient(hunting): get_auth_request con 0x7f9d7813e2e0 auth_method 0 2023-07-29T14:05:49.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d7cc11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d7cc11640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d7cc11640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d77fff640 10 monclient(hunting): get_auth_request con 0x7f9d780a86c0 auth_method 0 2023-07-29T14:05:49.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d77fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d77fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d7cc11640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d7cc11640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.311+0000 7f9d7cc11640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.312+0000 7f9d77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.312+0000 7f9d77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.312+0000 7f9d77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.313+0000 7f9d7cc11640 10 monclient(hunting): handle_auth_done global_id 16373 payload 274 2023-07-29T14:05:49.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.313+0000 7f9d7cc11640 1 -- 172.21.15.40:0/750480108 >> v2:172.21.15.40:3300/0 conn(0x7f9d780a86c0 msgr2=0x7f9d78140ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:49.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.313+0000 7f9d7cc11640 1 --2- 172.21.15.40:0/750480108 >> v2:172.21.15.40:3300/0 conn(0x7f9d780a86c0 0x7f9d78140ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.313+0000 7f9d7cc11640 1 -- 172.21.15.40:0/750480108 >> v1:172.21.15.40:6789/0 conn(0x7f9d781410e0 legacy=0x7f9d7813dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.313+0000 7f9d7cc11640 10 monclient: _finish_hunting 0 2023-07-29T14:05:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.313+0000 7f9d7cc11640 1 monclient: found mon.noname-b 2023-07-29T14:05:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.313+0000 7f9d7cc11640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.313+0000 7f9d7cc11640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.313+0000 7f9d7cc11640 1 -- 172.21.15.40:0/750480108 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7813e6e0 con 0x7f9d7813e2e0 2023-07-29T14:05:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.315+0000 7f9d7cc11640 1 --2- 172.21.15.40:0/750480108 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d7813e2e0 0x7f9d7814b4b0 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009e50 tx=0x7f9d7002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afa58e27fd9c50ec server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.317+0000 7f9d76ffd640 1 -- 172.21.15.40:0/750480108 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7002ca70 con 0x7f9d7813e2e0 2023-07-29T14:05:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.317+0000 7f9d76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.317+0000 7f9d76ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.317+0000 7f9d76ffd640 10 monclient: dump: 2023-07-29T14:05:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.326 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.326 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.326 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.326 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.327 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.317+0000 7f9d76ffd640 10 monclient: _finish_auth 0 2023-07-29T14:05:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.317+0000 7f9d76ffd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.317+0000 7f9d76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.317+0000 7f9d7e674640 5 monclient: authenticate success, global_id 16373 2023-07-29T14:05:49.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.317+0000 7f9d7e674640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T14:05:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.319+0000 7f9d76ffd640 1 -- 172.21.15.40:0/750480108 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7002cbd0 con 0x7f9d7813e2e0 2023-07-29T14:05:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.319+0000 7f9d76ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.319+0000 7f9d76ffd640 1 -- 172.21.15.40:0/750480108 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d700343f0 con 0x7f9d7813e2e0 2023-07-29T14:05:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.319+0000 7f9d76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.319+0000 7f9d76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.319+0000 7f9d76ffd640 10 monclient: dump: 2023-07-29T14:05:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.333 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.333 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.333 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.333 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.333 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.333 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.333 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.319+0000 7f9d7e674640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.319+0000 7f9d7e674640 4 set_mon_vals no callback set 2023-07-29T14:05:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.320+0000 7f9d7e674640 10 monclient: shutdown 2023-07-29T14:05:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.320+0000 7f9d7e674640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.320+0000 7f9d7e674640 1 -- 172.21.15.40:0/750480108 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d7813e2e0 msgr2=0x7f9d7814b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.320+0000 7f9d7e674640 1 --2- 172.21.15.40:0/750480108 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d7813e2e0 0x7f9d7814b4b0 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009e50 tx=0x7f9d7002bed0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.320+0000 7f9d7e674640 1 -- 172.21.15.40:0/750480108 shutdown_connections 2023-07-29T14:05:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.320+0000 7f9d7e674640 1 --2- 172.21.15.40:0/750480108 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d7813e2e0 0x7f9d7814b4b0 unknown :-1 s=CLOSED pgs=3223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.320+0000 7f9d7e674640 1 --2- 172.21.15.40:0/750480108 >> v2:172.21.15.40:3300/0 conn(0x7f9d780a86c0 0x7f9d78140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.320+0000 7f9d7e674640 1 -- 172.21.15.40:0/750480108 >> 172.21.15.40:0/750480108 conn(0x7f9d78065920 msgr2=0x7f9d78065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.320+0000 7f9d7e674640 1 -- 172.21.15.40:0/750480108 shutdown_connections 2023-07-29T14:05:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.320+0000 7f9d7e674640 1 -- 172.21.15.40:0/750480108 wait complete. 2023-07-29T14:05:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.321+0000 7f9d7e674640 10 monclient: build_initial_monmap 2023-07-29T14:05:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.321+0000 7f9d7e674640 10 monclient: monmap: 2023-07-29T14:05:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:49.337 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.321+0000 7f9d7e674640 1 librados: starting msgr at 2023-07-29T14:05:49.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.321+0000 7f9d7e674640 1 librados: starting objecter 2023-07-29T14:05:49.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.321+0000 7f9d7e674640 1 Processor -- start 2023-07-29T14:05:49.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.321+0000 7f9d7e674640 1 -- start start 2023-07-29T14:05:49.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.321+0000 7f9d7e674640 1 librados: setting wanted keys 2023-07-29T14:05:49.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.321+0000 7f9d7e674640 1 librados: calling monclient init 2023-07-29T14:05:49.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.321+0000 7f9d7e674640 10 monclient: init 2023-07-29T14:05:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T14:05:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d780a86c0 0x7f9d78143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 10 monclient(hunting): picked mon.noname-a con 0x7f9d780a86c0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d781410e0 0x7f9d781383f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 10 monclient(hunting): picked mon.noname-c con 0x7f9d781410e0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:49.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d78141f20 0x7f9d78144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 10 monclient(hunting): picked mon.noname-b con 0x7f9d78141f20 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:49.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d78138930 con 0x7f9d780a86c0 2023-07-29T14:05:49.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780aad20 con 0x7f9d78141f20 2023-07-29T14:05:49.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780ab000 con 0x7f9d781410e0 2023-07-29T14:05:49.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:49.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d7e674640 10 monclient(hunting): authenticate will time out at 3671.454518s 2023-07-29T14:05:49.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.322+0000 7f9d777fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d781410e0 0x7f9d781383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d777fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d781410e0 0x7f9d781383f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49536/0 (socket says 172.21.15.40:49536) 2023-07-29T14:05:49.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d777fe640 1 -- 172.21.15.40:0/3161575900 learned_addr learned my addr 172.21.15.40:0/3161575900 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:49.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d777fe640 10 monclient(hunting): get_auth_request con 0x7f9d781410e0 auth_method 0 2023-07-29T14:05:49.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d777fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d777fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d7cc11640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d78141f20 0x7f9d78144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d7cc11640 10 monclient(hunting): get_auth_request con 0x7f9d78141f20 auth_method 0 2023-07-29T14:05:49.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d7cc11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d7cc11640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d7cc11640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.323+0000 7f9d77fff640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d780a86c0 0x7f9d78143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.324+0000 7f9d7cc11640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.324+0000 7f9d7cc11640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.324+0000 7f9d7cc11640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.324+0000 7f9d77fff640 10 monclient(hunting): get_auth_request con 0x7f9d780a86c0 auth_method 0 2023-07-29T14:05:49.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.324+0000 7f9d77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.324+0000 7f9d77fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.324+0000 7f9d77fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.324+0000 7f9d77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.324+0000 7f9d77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.324+0000 7f9d77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.326+0000 7f9d777fe640 10 monclient(hunting): handle_auth_done global_id 16376 payload 824 2023-07-29T14:05:49.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.326+0000 7f9d777fe640 1 -- 172.21.15.40:0/3161575900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d78141f20 msgr2=0x7f9d78144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.326+0000 7f9d777fe640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d78141f20 0x7f9d78144680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.326+0000 7f9d777fe640 1 -- 172.21.15.40:0/3161575900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d780a86c0 msgr2=0x7f9d78143ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.326+0000 7f9d777fe640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d780a86c0 0x7f9d78143ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.326+0000 7f9d777fe640 10 monclient: _finish_hunting 0 2023-07-29T14:05:49.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.326+0000 7f9d777fe640 1 monclient: found mon.noname-c 2023-07-29T14:05:49.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.326+0000 7f9d777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:49.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.326+0000 7f9d777fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.326+0000 7f9d777fe640 1 -- 172.21.15.40:0/3161575900 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70009b30 con 0x7f9d781410e0 2023-07-29T14:05:49.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.328+0000 7f9d777fe640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d781410e0 0x7f9d781383f0 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000c3c0 tx=0x7f9d6000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:49.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.328+0000 7f9d74ff9640 10 client.?.objecter ms_handle_connect 0x7f9d781410e0 2023-07-29T14:05:49.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.328+0000 7f9d74ff9640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:49.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 1 -- 172.21.15.40:0/3161575900 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d600052d0 con 0x7f9d781410e0 2023-07-29T14:05:49.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:49.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 10 monclient: dump: 2023-07-29T14:05:49.356 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.356 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.356 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.356 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.357 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.357 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.357 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.357 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.357 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.357 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 10 monclient: _finish_auth 0 2023-07-29T14:05:49.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 10 monclient: _check_auth_tickets 2023-07-29T14:05:49.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:49.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 1 -- 172.21.15.40:0/3161575900 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d60005450 con 0x7f9d781410e0 2023-07-29T14:05:49.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d7e674640 5 monclient: authenticate success, global_id 16376 2023-07-29T14:05:49.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d7e674640 10 monclient: _renew_subs 2023-07-29T14:05:49.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d7e674640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d7e674640 1 -- 172.21.15.40:0/3161575900 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d78144bc0 con 0x7f9d781410e0 2023-07-29T14:05:49.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:49.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 1 -- 172.21.15.40:0/3161575900 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60007260 con 0x7f9d781410e0 2023-07-29T14:05:49.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:49.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d74ff9640 10 monclient: dump: 2023-07-29T14:05:49.359 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.359 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.359 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.360 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.360 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.360 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.361 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.361 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.362 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.362 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d767fc640 4 set_mon_vals no callback set 2023-07-29T14:05:49.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d7e674640 10 client.16376.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:49.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d7e674640 10 monclient: _renew_subs 2023-07-29T14:05:49.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d7e674640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d7e674640 1 -- 172.21.15.40:0/3161575900 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d781d92b0 con 0x7f9d781410e0 2023-07-29T14:05:49.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.332+0000 7f9d7e674640 1 librados: init done 2023-07-29T14:05:49.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.333+0000 7f9d7e674640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:49.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.333+0000 7f9d7e674640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:49.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.333+0000 7f9d7e674640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.333+0000 7f9d7e674640 1 -- 172.21.15.40:0/3161575900 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d78138aa0 con 0x7f9d781410e0 2023-07-29T14:05:49.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.353+0000 7f9d74ff9640 1 -- 172.21.15.40:0/3161575900 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f9d600073c0 con 0x7f9d781410e0 2023-07-29T14:05:49.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.353+0000 7f9d74ff9640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f9d5806bfb0 0x7f9d5806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.354+0000 7f9d77fff640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f9d5806bfb0 0x7f9d5806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.354+0000 7f9d77fff640 10 monclient: get_auth_request con 0x7f9d5806bfb0 auth_method 0 2023-07-29T14:05:49.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.354+0000 7f9d77fff640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f9d5806bfb0 0x7f9d5806e460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009b40 tx=0x7f9d68009190 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:49.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.355+0000 7f9d74ff9640 10 client.16376.objecter ms_handle_connect 0x7f9d5806bfb0 2023-07-29T14:05:49.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.355+0000 7f9d74ff9640 1 -- 172.21.15.40:0/3161575900 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f9d6005b3c0 con 0x7f9d781410e0 2023-07-29T14:05:49.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.355+0000 7f9d74ff9640 10 client.16376.objecter ms_dispatch 0x7f9d78000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:49.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.355+0000 7f9d74ff9640 3 client.16376.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:49.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.355+0000 7f9d74ff9640 3 client.16376.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:49.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.356+0000 7f9d74ff9640 20 client.16376.objecter dump_active .. 0 homeless 2023-07-29T14:05:49.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.605+0000 7f9d74ff9640 1 -- 172.21.15.40:0/3161575900 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f9d600537f0 con 0x7f9d781410e0 2023-07-29T14:05:49.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.605+0000 7f9d74ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:49.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.605+0000 7f9d74ff9640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:49.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.784+0000 7f9d7e674640 1 -- 172.21.15.40:0/3161575900 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d780b2a40 con 0x7f9d5806bfb0 2023-07-29T14:05:49.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.785+0000 7f9d74ff9640 1 -- 172.21.15.40:0/3161575900 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9d68011a60 con 0x7f9d5806bfb0 2023-07-29T14:05:49.785 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2023-07-29T14:05:49.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 10 librados: watch_flush enter 2023-07-29T14:05:49.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 10 librados: watch_flush exit 2023-07-29T14:05:49.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 20 client.16376.objecter shutdown clearing up homeless session... 2023-07-29T14:05:49.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 10 client.16376.objecter successfully canceled tick 2023-07-29T14:05:49.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 -- 172.21.15.40:0/3161575900 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f9d5806bfb0 msgr2=0x7f9d5806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f9d5806bfb0 0x7f9d5806e460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009b40 tx=0x7f9d68009190 comp rx=0 tx=0).stop 2023-07-29T14:05:49.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 10 monclient: shutdown 2023-07-29T14:05:49.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:49.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 -- 172.21.15.40:0/3161575900 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d781410e0 msgr2=0x7f9d781383f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d781410e0 0x7f9d781383f0 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000c3c0 tx=0x7f9d6000c880 comp rx=0 tx=0).stop 2023-07-29T14:05:49.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 -- 172.21.15.40:0/3161575900 shutdown_connections 2023-07-29T14:05:49.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f9d5806bfb0 0x7f9d5806e460 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d78141f20 0x7f9d78144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9d781410e0 0x7f9d781383f0 unknown :-1 s=CLOSED pgs=3224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 --2- 172.21.15.40:0/3161575900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d780a86c0 0x7f9d78143ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 -- 172.21.15.40:0/3161575900 >> 172.21.15.40:0/3161575900 conn(0x7f9d78065920 msgr2=0x7f9d780b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:49.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 -- 172.21.15.40:0/3161575900 shutdown_connections 2023-07-29T14:05:49.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 -- 172.21.15.40:0/3161575900 wait complete. 2023-07-29T14:05:49.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.787+0000 7f9d7e674640 1 librados: shutdown 2023-07-29T14:05:49.803 INFO:tasks.ceph:Scrubbing osd.2 2023-07-29T14:05:49.803 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-07-29T14:05:49.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.913+0000 7f79ef299640 10 monclient: get_monmap_and_config 2023-07-29T14:05:49.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.913+0000 7f79ef299640 10 monclient: build_initial_monmap 2023-07-29T14:05:49.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.913+0000 7f79ef299640 10 monclient: monmap: 2023-07-29T14:05:49.914 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:49.914 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.914 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:49.913788+0000 2023-07-29T14:05:49.914 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:49.913788+0000 2023-07-29T14:05:49.914 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:49.915 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.915 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:49.915 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:49.915 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:49.915 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:49.915 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:49.915 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 1 Processor -- start 2023-07-29T14:05:49.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 1 -- start start 2023-07-29T14:05:49.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 10 monclient: init 2023-07-29T14:05:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2023-07-29T14:05:49.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 0x7f79e80b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 10 monclient(hunting): picked mon.noname-b con 0x7f79e80b3a70 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:49.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f79e813dda0 0x7f79e813e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 10 monclient(hunting): picked mon.noname-c con 0x7f79e813dda0 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:49.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79e81442b0 0x7f79e8144730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 10 monclient(hunting): picked mon.noname-a con 0x7f79e81442b0 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:49.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e80b3e40 con 0x7f79e81442b0 2023-07-29T14:05:49.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e80a8d80 con 0x7f79e813dda0 2023-07-29T14:05:49.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e80a97d0 con 0x7f79e80b3a70 2023-07-29T14:05:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.914+0000 7f79ef299640 10 monclient(hunting): authenticate will time out at 3672.047060s 2023-07-29T14:05:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.915+0000 7f79ed035640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 0x7f79e80b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.915+0000 7f79ed836640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79e81442b0 0x7f79e8144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.915+0000 7f79ed035640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 0x7f79e80b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49548/0 (socket says 172.21.15.40:49548) 2023-07-29T14:05:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.915+0000 7f79ec834640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f79e813dda0 0x7f79e813e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.915+0000 7f79ed035640 1 -- 172.21.15.40:0/4240849132 learned_addr learned my addr 172.21.15.40:0/4240849132 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.915+0000 7f79ed035640 10 monclient(hunting): get_auth_request con 0x7f79e80b3a70 auth_method 0 2023-07-29T14:05:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.915+0000 7f79ed035640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.915+0000 7f79ed035640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.915+0000 7f79ed035640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ed836640 10 monclient(hunting): get_auth_request con 0x7f79e81442b0 auth_method 0 2023-07-29T14:05:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ed836640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ed836640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ed836640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ec834640 10 monclient(hunting): get_auth_request con 0x7f79e813dda0 auth_method 0 2023-07-29T14:05:49.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ec834640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ec834640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ec834640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ed035640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ed035640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ed035640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ed836640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ed836640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ed836640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ec834640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ec834640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.916+0000 7f79ec834640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.918+0000 7f79ed035640 10 monclient(hunting): handle_auth_done global_id 16379 payload 274 2023-07-29T14:05:49.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.918+0000 7f79ed035640 1 -- 172.21.15.40:0/4240849132 >> v2:172.21.15.45:3300/0 conn(0x7f79e813dda0 msgr2=0x7f79e813e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:49.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.918+0000 7f79ed035640 1 --2- 172.21.15.40:0/4240849132 >> v2:172.21.15.45:3300/0 conn(0x7f79e813dda0 0x7f79e813e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.918+0000 7f79ed035640 1 -- 172.21.15.40:0/4240849132 >> v2:172.21.15.40:3300/0 conn(0x7f79e81442b0 msgr2=0x7f79e8144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.918+0000 7f79ed035640 1 --2- 172.21.15.40:0/4240849132 >> v2:172.21.15.40:3300/0 conn(0x7f79e81442b0 0x7f79e8144730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.918+0000 7f79ed035640 10 monclient: _finish_hunting 0 2023-07-29T14:05:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.918+0000 7f79ed035640 1 monclient: found mon.noname-b 2023-07-29T14:05:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.918+0000 7f79ed035640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.918+0000 7f79ed035640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.918+0000 7f79ed035640 1 -- 172.21.15.40:0/4240849132 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e813e6f0 con 0x7f79e80b3a70 2023-07-29T14:05:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.921+0000 7f79ed035640 1 --2- 172.21.15.40:0/4240849132 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 0x7f79e80b2b10 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 crypto rx=0x7f79e8071f60 tx=0x7f79e402b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d7c1f29c9d44282 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:49.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 1 -- 172.21.15.40:0/4240849132 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4031750 con 0x7f79e80b3a70 2023-07-29T14:05:49.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:49.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 10 monclient: dump: 2023-07-29T14:05:49.941 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.941 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.941 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.941 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.941 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.941 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.941 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.942 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.942 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.942 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 10 monclient: _finish_auth 0 2023-07-29T14:05:49.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 10 monclient: _check_auth_tickets 2023-07-29T14:05:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 1 -- 172.21.15.40:0/4240849132 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e40318b0 con 0x7f79e80b3a70 2023-07-29T14:05:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 1 -- 172.21.15.40:0/4240849132 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4031c40 con 0x7f79e80b3a70 2023-07-29T14:05:49.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 5 monclient: authenticate success, global_id 16379 2023-07-29T14:05:49.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:49.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 4 set_mon_vals no callback set 2023-07-29T14:05:49.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:49.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79dffff640 10 monclient: dump: 2023-07-29T14:05:49.944 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.945 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.945 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.945 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.945 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.945 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.945 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.945 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.945 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.946 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 10 monclient: shutdown 2023-07-29T14:05:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 1 -- 172.21.15.40:0/4240849132 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 msgr2=0x7f79e80b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 1 --2- 172.21.15.40:0/4240849132 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 0x7f79e80b2b10 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 crypto rx=0x7f79e8071f60 tx=0x7f79e402b7f0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 1 -- 172.21.15.40:0/4240849132 shutdown_connections 2023-07-29T14:05:49.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 1 --2- 172.21.15.40:0/4240849132 >> v2:172.21.15.45:3300/0 conn(0x7f79e813dda0 0x7f79e813e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 1 --2- 172.21.15.40:0/4240849132 >> v2:172.21.15.40:3300/0 conn(0x7f79e81442b0 0x7f79e8144730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 1 --2- 172.21.15.40:0/4240849132 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 0x7f79e80b2b10 unknown :-1 s=CLOSED pgs=3225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.925+0000 7f79ef299640 1 -- 172.21.15.40:0/4240849132 >> 172.21.15.40:0/4240849132 conn(0x7f79e8071720 msgr2=0x7f79e8065920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.926+0000 7f79ef299640 1 -- 172.21.15.40:0/4240849132 shutdown_connections 2023-07-29T14:05:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.926+0000 7f79ef299640 1 -- 172.21.15.40:0/4240849132 wait complete. 2023-07-29T14:05:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.926+0000 7f79ef299640 10 monclient: build_initial_monmap 2023-07-29T14:05:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.926+0000 7f79ef299640 10 monclient: monmap: 2023-07-29T14:05:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:49.953 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:49.953 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:49.953 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:49.953 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.953 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:49.953 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:49.953 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:49.953 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.926+0000 7f79ef299640 1 librados: starting msgr at 2023-07-29T14:05:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.926+0000 7f79ef299640 1 librados: starting objecter 2023-07-29T14:05:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 1 Processor -- start 2023-07-29T14:05:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 1 -- start start 2023-07-29T14:05:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 1 librados: setting wanted keys 2023-07-29T14:05:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 1 librados: calling monclient init 2023-07-29T14:05:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 10 monclient: init 2023-07-29T14:05:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T14:05:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 0x7f79e80ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 10 monclient(hunting): picked mon.noname-c con 0x7f79e80b3a70 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79e813dda0 0x7f79e80ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 10 monclient(hunting): picked mon.noname-a con 0x7f79e813dda0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79e81442b0 0x7f79e81d60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 10 monclient(hunting): picked mon.noname-b con 0x7f79e81442b0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:49.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e81d5c20 con 0x7f79e813dda0 2023-07-29T14:05:49.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e80af210 con 0x7f79e81442b0 2023-07-29T14:05:49.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:49.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e80af3d0 con 0x7f79e80b3a70 2023-07-29T14:05:49.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:49.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.927+0000 7f79ef299640 10 monclient(hunting): authenticate will time out at 3672.059799s 2023-07-29T14:05:49.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed035640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 0x7f79e80ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ec834640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79e813dda0 0x7f79e80ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed035640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 0x7f79e80ac790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:49560/0 (socket says 172.21.15.40:49560) 2023-07-29T14:05:49.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed035640 1 -- 172.21.15.40:0/3694841005 learned_addr learned my addr 172.21.15.40:0/3694841005 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:49.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed035640 10 monclient(hunting): get_auth_request con 0x7f79e80b3a70 auth_method 0 2023-07-29T14:05:49.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed035640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed035640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed035640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed836640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79e81442b0 0x7f79e81d60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ec834640 10 monclient(hunting): get_auth_request con 0x7f79e813dda0 auth_method 0 2023-07-29T14:05:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ec834640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ec834640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ec834640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed836640 10 monclient(hunting): get_auth_request con 0x7f79e81442b0 auth_method 0 2023-07-29T14:05:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed836640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed836640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed836640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed035640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed035640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.928+0000 7f79ed035640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.929+0000 7f79ec834640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.929+0000 7f79ec834640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.929+0000 7f79ec834640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.929+0000 7f79ed836640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:49.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.929+0000 7f79ed836640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:49.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.929+0000 7f79ed836640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:49.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.931+0000 7f79ed035640 10 monclient(hunting): handle_auth_done global_id 16382 payload 824 2023-07-29T14:05:49.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.931+0000 7f79ed035640 1 -- 172.21.15.40:0/3694841005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79e81442b0 msgr2=0x7f79e81d60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.931+0000 7f79ed035640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79e81442b0 0x7f79e81d60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.931+0000 7f79ed035640 1 -- 172.21.15.40:0/3694841005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79e813dda0 msgr2=0x7f79e80ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:49.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.931+0000 7f79ed035640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79e813dda0 0x7f79e80ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:49.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.931+0000 7f79ed035640 10 monclient: _finish_hunting 0 2023-07-29T14:05:49.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.931+0000 7f79ed035640 1 monclient: found mon.noname-c 2023-07-29T14:05:49.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.931+0000 7f79ed035640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:49.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.932+0000 7f79ed035640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.932+0000 7f79ed035640 1 -- 172.21.15.40:0/3694841005 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e40094d0 con 0x7f79e80b3a70 2023-07-29T14:05:49.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.934+0000 7f79ed035640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 0x7f79e80ac790 secure :-1 s=READY pgs=3226 cs=0 l=1 rev1=1 crypto rx=0x7f79e4000c00 tx=0x7f79e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:49.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.934+0000 7f79ddffb640 10 client.?.objecter ms_handle_connect 0x7f79e80b3a70 2023-07-29T14:05:49.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.934+0000 7f79ddffb640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:49.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 1 -- 172.21.15.40:0/3694841005 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40338d0 con 0x7f79e80b3a70 2023-07-29T14:05:49.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:49.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 10 monclient: dump: 2023-07-29T14:05:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.971 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.971 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.971 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.971 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.971 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 10 monclient: _finish_auth 0 2023-07-29T14:05:49.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 10 monclient: _check_auth_tickets 2023-07-29T14:05:49.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:49.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 1 -- 172.21.15.40:0/3694841005 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e4033a30 con 0x7f79e80b3a70 2023-07-29T14:05:49.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ef299640 5 monclient: authenticate success, global_id 16382 2023-07-29T14:05:49.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:49.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ef299640 10 monclient: _renew_subs 2023-07-29T14:05:49.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 1 -- 172.21.15.40:0/3694841005 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4033dc0 con 0x7f79e80b3a70 2023-07-29T14:05:49.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ef299640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ef299640 1 -- 172.21.15.40:0/3694841005 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e80ada70 con 0x7f79e80b3a70 2023-07-29T14:05:49.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:49.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79df7fe640 4 set_mon_vals no callback set 2023-07-29T14:05:49.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:49.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ddffb640 10 monclient: dump: 2023-07-29T14:05:49.974 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:49.974 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:49.974 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.974 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:49.974 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:49.974 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:49.975 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:49.975 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:49.975 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:49.978 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:49.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ef299640 10 client.16382.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:49.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ef299640 10 monclient: _renew_subs 2023-07-29T14:05:49.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ef299640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ef299640 1 -- 172.21.15.40:0/3694841005 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e81d65f0 con 0x7f79e80b3a70 2023-07-29T14:05:49.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.938+0000 7f79ef299640 1 librados: init done 2023-07-29T14:05:49.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.939+0000 7f79ef299640 10 client.16382.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:49.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.939+0000 7f79ef299640 15 client.16382.objecter _session_command_op_assign -1 1 2023-07-29T14:05:49.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.939+0000 7f79ef299640 20 client.16382.objecter _get_session osd=-1 returning homeless 2023-07-29T14:05:49.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.939+0000 7f79ef299640 10 client.16382.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:49.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.939+0000 7f79ef299640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:49.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.939+0000 7f79ef299640 1 -- 172.21.15.40:0/3694841005 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79e80a9d90 con 0x7f79e80b3a70 2023-07-29T14:05:49.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.958+0000 7f79ddffb640 1 -- 172.21.15.40:0/3694841005 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f79e403a040 con 0x7f79e80b3a70 2023-07-29T14:05:49.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.958+0000 7f79ddffb640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f79cc06bfb0 0x7f79cc06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.959+0000 7f79ec834640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f79cc06bfb0 0x7f79cc06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.959+0000 7f79ec834640 10 monclient: get_auth_request con 0x7f79cc06bfb0 auth_method 0 2023-07-29T14:05:49.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.959+0000 7f79ec834640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f79cc06bfb0 0x7f79cc06e460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f79e80ae760 tx=0x7f79d8009190 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:49.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.959+0000 7f79ddffb640 10 client.16382.objecter ms_handle_connect 0x7f79cc06bfb0 2023-07-29T14:05:49.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 1 -- 172.21.15.40:0/3694841005 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f79e407df70 con 0x7f79e80b3a70 2023-07-29T14:05:49.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 10 client.16382.objecter ms_dispatch 0x7f79e8000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:49.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 3 client.16382.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:49.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 3 client.16382.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:49.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 10 client.16382.objecter checking command 1 2023-07-29T14:05:49.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x7f79cc0737f0 0x7f79cc075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:49.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 20 client.16382.objecter _get_session s=0x7f79cc073120 osd=2 3 2023-07-29T14:05:49.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 20 client.16382.objecter put_session s=0x7f79cc073120 osd=2 3 2023-07-29T14:05:49.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 15 client.16382.objecter _session_command_op_remove -1 1 2023-07-29T14:05:49.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 20 client.16382.objecter _get_session s=0x7f79cc073120 osd=2 3 2023-07-29T14:05:49.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 20 client.16382.objecter get_session s=0x7f79cc073120 osd=2 3 2023-07-29T14:05:49.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 15 client.16382.objecter _session_command_op_assign 2 1 2023-07-29T14:05:49.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 20 client.16382.objecter put_session s=0x7f79cc073120 osd=2 4 2023-07-29T14:05:49.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 10 client.16382.objecter _send_command 1 2023-07-29T14:05:49.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 1 -- 172.21.15.40:0/3694841005 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79cc076350 con 0x7f79cc0737f0 2023-07-29T14:05:49.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 20 client.16382.objecter dump_active .. 0 homeless 2023-07-29T14:05:49.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 1 -- 172.21.15.40:0/3694841005 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=1004) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79e40763a0 con 0x7f79e80b3a70 2023-07-29T14:05:49.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.962+0000 7f79ddffb640 10 monclient: handle_get_version_reply finishing 1 version 1004 2023-07-29T14:05:49.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.963+0000 7f79ed836640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x7f79cc0737f0 0x7f79cc075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:49.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.963+0000 7f79ed836640 10 monclient: get_auth_request con 0x7f79cc0737f0 auth_method 0 2023-07-29T14:05:49.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.966+0000 7f79ed836640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x7f79cc0737f0 0x7f79cc075c80 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:49.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.966+0000 7f79ddffb640 10 client.16382.objecter ms_handle_connect 0x7f79cc0737f0 2023-07-29T14:05:49.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.990+0000 7f79ddffb640 1 -- 172.21.15.40:0/3694841005 <== osd.2 v2:172.21.15.40:6800/85572 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f79cc076350 con 0x7f79cc0737f0 2023-07-29T14:05:49.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.990+0000 7f79ddffb640 10 client.16382.objecter ms_dispatch 0x7f79e8000b90 command_reply(tid 1: 0 ) v1 2023-07-29T14:05:49.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.990+0000 7f79ddffb640 10 client.16382.objecter _finish_command 1 = system:0 2023-07-29T14:05:49.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.990+0000 7f79ddffb640 20 client.16382.objecter put_session s=0x7f79cc073120 osd=2 4 2023-07-29T14:05:49.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:49.990+0000 7f79ddffb640 15 client.16382.objecter _session_command_op_remove 2 1 2023-07-29T14:05:50.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.003+0000 7f79ef299640 10 client.16382.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-07-29T14:05:50.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.003+0000 7f79ef299640 15 client.16382.objecter _session_command_op_assign -1 2 2023-07-29T14:05:50.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.003+0000 7f79ef299640 20 client.16382.objecter _get_session s=0x7f79cc073120 osd=2 3 2023-07-29T14:05:50.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.003+0000 7f79ef299640 20 client.16382.objecter put_session s=0x7f79cc073120 osd=2 3 2023-07-29T14:05:50.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.003+0000 7f79ef299640 20 client.16382.objecter _get_session s=0x7f79cc073120 osd=2 3 2023-07-29T14:05:50.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.003+0000 7f79ef299640 15 client.16382.objecter _session_command_op_remove -1 2 2023-07-29T14:05:50.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.003+0000 7f79ef299640 20 client.16382.objecter get_session s=0x7f79cc073120 osd=2 3 2023-07-29T14:05:50.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.003+0000 7f79ef299640 15 client.16382.objecter _session_command_op_assign 2 2 2023-07-29T14:05:50.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.003+0000 7f79ef299640 20 client.16382.objecter put_session s=0x7f79cc073120 osd=2 4 2023-07-29T14:05:50.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.003+0000 7f79ef299640 10 client.16382.objecter _send_command 2 2023-07-29T14:05:50.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.003+0000 7f79ef299640 1 -- 172.21.15.40:0/3694841005 --> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f79e80a9ab0 con 0x7f79cc0737f0 2023-07-29T14:05:50.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ddffb640 1 -- 172.21.15.40:0/3694841005 <== osd.2 v2:172.21.15.40:6800/85572 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f79e000f040 con 0x7f79cc0737f0 2023-07-29T14:05:50.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ddffb640 10 client.16382.objecter ms_dispatch 0x7f79e8000b90 command_reply(tid 2: 0 ) v1 2023-07-29T14:05:50.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ddffb640 10 client.16382.objecter _finish_command 2 = system:0 2023-07-29T14:05:50.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ddffb640 20 client.16382.objecter put_session s=0x7f79cc073120 osd=2 4 2023-07-29T14:05:50.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ddffb640 15 client.16382.objecter _session_command_op_remove 2 2 2023-07-29T14:05:50.030 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-29T14:05:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-29T14:05:50.031 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-29T14:05:50.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 10 librados: watch_flush enter 2023-07-29T14:05:50.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 10 librados: watch_flush exit 2023-07-29T14:05:50.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 10 client.16382.objecter close_session for osd.2 2023-07-29T14:05:50.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 1 -- 172.21.15.40:0/3694841005 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x7f79cc0737f0 msgr2=0x7f79cc075c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.40:6800/85572,v1:172.21.15.40:6801/85572] conn(0x7f79cc0737f0 0x7f79cc075c80 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 20 client.16382.objecter put_session s=0x7f79cc073120 osd=2 1 2023-07-29T14:05:50.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 20 client.16382.objecter shutdown clearing up homeless session... 2023-07-29T14:05:50.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 10 client.16382.objecter successfully canceled tick 2023-07-29T14:05:50.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 1 -- 172.21.15.40:0/3694841005 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f79cc06bfb0 msgr2=0x7f79cc06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f79cc06bfb0 0x7f79cc06e460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f79e80ae760 tx=0x7f79d8009190 comp rx=0 tx=0).stop 2023-07-29T14:05:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 10 monclient: shutdown 2023-07-29T14:05:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 1 -- 172.21.15.40:0/3694841005 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 msgr2=0x7f79e80ac790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 1 --2- 172.21.15.40:0/3694841005 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f79e80b3a70 0x7f79e80ac790 secure :-1 s=READY pgs=3226 cs=0 l=1 rev1=1 crypto rx=0x7f79e4000c00 tx=0x7f79e40046c0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ed836640 1 -- 172.21.15.40:0/3694841005 reap_dead start 2023-07-29T14:05:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 1 -- 172.21.15.40:0/3694841005 shutdown_connections 2023-07-29T14:05:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 1 -- 172.21.15.40:0/3694841005 >> 172.21.15.40:0/3694841005 conn(0x7f79e8071720 msgr2=0x7f79e8068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 1 -- 172.21.15.40:0/3694841005 shutdown_connections 2023-07-29T14:05:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 1 -- 172.21.15.40:0/3694841005 wait complete. 2023-07-29T14:05:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.017+0000 7f79ef299640 1 librados: shutdown 2023-07-29T14:05:50.035 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-07-29T14:05:50.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.146+0000 7f46acfb2640 10 monclient: get_monmap_and_config 2023-07-29T14:05:50.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.146+0000 7f46acfb2640 10 monclient: build_initial_monmap 2023-07-29T14:05:50.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.146+0000 7f46acfb2640 10 monclient: monmap: 2023-07-29T14:05:50.150 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:50.150 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:50.150 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:50.147095+0000 2023-07-29T14:05:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:50.147095+0000 2023-07-29T14:05:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:50.152 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.147+0000 7f46acfb2640 1 Processor -- start 2023-07-29T14:05:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.147+0000 7f46acfb2640 1 -- start start 2023-07-29T14:05:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.147+0000 7f46acfb2640 10 monclient: init 2023-07-29T14:05:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2023-07-29T14:05:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f46a80b74a0 0x7f46a80aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 10 monclient(hunting): picked mon.noname-b con 0x7f46a80b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f46a813dde0 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46a81445e0 0x7f46a8144a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 10 monclient(hunting): picked mon.noname-a con 0x7f46a81445e0 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a80b7870 con 0x7f46a813dde0 2023-07-29T14:05:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a80a91b0 con 0x7f46a81445e0 2023-07-29T14:05:50.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:50.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80a9490 con 0x7f46a80b74a0 2023-07-29T14:05:50.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:50.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46acfb2640 10 monclient(hunting): authenticate will time out at 3672.280445s 2023-07-29T14:05:50.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46a659c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f46a80b74a0 0x7f46a80aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.148+0000 7f46a6d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46a81445e0 0x7f46a8144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a5d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f46a813dde0 0x7f46a813e1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.40:53836/0 (socket says 172.21.15.40:53836) 2023-07-29T14:05:50.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a5d9b640 1 -- 172.21.15.40:0/621480449 learned_addr learned my addr 172.21.15.40:0/621480449 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:50.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a659c640 10 monclient(hunting): get_auth_request con 0x7f46a80b74a0 auth_method 0 2023-07-29T14:05:50.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:50.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a659c640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:50.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a659c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:50.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a6d9d640 10 monclient(hunting): get_auth_request con 0x7f46a81445e0 auth_method 0 2023-07-29T14:05:50.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:50.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a6d9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:50.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a6d9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:50.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:50.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:50.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:50.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:50.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:50.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.149+0000 7f46a6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:50.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.151+0000 7f46a659c640 10 monclient(hunting): handle_auth_done global_id 16385 payload 274 2023-07-29T14:05:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.151+0000 7f46a659c640 1 -- 172.21.15.40:0/621480449 >> v2:172.21.15.40:3300/0 conn(0x7f46a81445e0 msgr2=0x7f46a8144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.151+0000 7f46a659c640 1 --2- 172.21.15.40:0/621480449 >> v2:172.21.15.40:3300/0 conn(0x7f46a81445e0 0x7f46a8144a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.151+0000 7f46a659c640 1 -- 172.21.15.40:0/621480449 >> v1:172.21.15.45:6789/0 conn(0x7f46a813dde0 legacy=0x7f46a813e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.151+0000 7f46a659c640 10 monclient: _finish_hunting 0 2023-07-29T14:05:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.151+0000 7f46a659c640 1 monclient: found mon.noname-b 2023-07-29T14:05:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.151+0000 7f46a659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.151+0000 7f46a659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.151+0000 7f46a659c640 1 -- 172.21.15.40:0/621480449 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a80a9770 con 0x7f46a80b74a0 2023-07-29T14:05:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.153+0000 7f46a659c640 1 --2- 172.21.15.40:0/621480449 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f46a80b74a0 0x7f46a80aa150 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009790 tx=0x7f46a002b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25707f6f63f2b0e server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 1 -- 172.21.15.40:0/621480449 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0031770 con 0x7f46a80b74a0 2023-07-29T14:05:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 10 monclient: dump: 2023-07-29T14:05:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:50.171 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 10 monclient: _finish_auth 0 2023-07-29T14:05:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 10 monclient: _check_auth_tickets 2023-07-29T14:05:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 1 -- 172.21.15.40:0/621480449 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a00318d0 con 0x7f46a80b74a0 2023-07-29T14:05:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 1 -- 172.21.15.40:0/621480449 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0031c60 con 0x7f46a80b74a0 2023-07-29T14:05:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46a559a640 10 monclient: dump: 2023-07-29T14:05:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:50.174 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46acfb2640 5 monclient: authenticate success, global_id 16385 2023-07-29T14:05:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46acfb2640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.157+0000 7f46acfb2640 4 set_mon_vals no callback set 2023-07-29T14:05:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.158+0000 7f46acfb2640 10 monclient: shutdown 2023-07-29T14:05:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.158+0000 7f46acfb2640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.158+0000 7f46acfb2640 1 -- 172.21.15.40:0/621480449 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f46a80b74a0 msgr2=0x7f46a80aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.158+0000 7f46acfb2640 1 --2- 172.21.15.40:0/621480449 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f46a80b74a0 0x7f46a80aa150 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009790 tx=0x7f46a002b810 comp rx=0 tx=0).stop 2023-07-29T14:05:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.158+0000 7f46acfb2640 1 -- 172.21.15.40:0/621480449 shutdown_connections 2023-07-29T14:05:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.158+0000 7f46acfb2640 1 --2- 172.21.15.40:0/621480449 >> v2:172.21.15.40:3300/0 conn(0x7f46a81445e0 0x7f46a8144a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.158+0000 7f46acfb2640 1 --2- 172.21.15.40:0/621480449 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f46a80b74a0 0x7f46a80aa150 unknown :-1 s=CLOSED pgs=3227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.158+0000 7f46acfb2640 1 -- 172.21.15.40:0/621480449 >> 172.21.15.40:0/621480449 conn(0x7f46a8065b00 msgr2=0x7f46a8065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.158+0000 7f46acfb2640 1 -- 172.21.15.40:0/621480449 shutdown_connections 2023-07-29T14:05:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.158+0000 7f46acfb2640 1 -- 172.21.15.40:0/621480449 wait complete. 2023-07-29T14:05:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.159+0000 7f46acfb2640 10 monclient: build_initial_monmap 2023-07-29T14:05:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.159+0000 7f46acfb2640 10 monclient: monmap: 2023-07-29T14:05:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:50.183 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.159+0000 7f46acfb2640 1 librados: starting msgr at 2023-07-29T14:05:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.159+0000 7f46acfb2640 1 librados: starting objecter 2023-07-29T14:05:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.159+0000 7f46acfb2640 1 Processor -- start 2023-07-29T14:05:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.159+0000 7f46acfb2640 1 -- start start 2023-07-29T14:05:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.159+0000 7f46acfb2640 1 librados: setting wanted keys 2023-07-29T14:05:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.159+0000 7f46acfb2640 1 librados: calling monclient init 2023-07-29T14:05:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.159+0000 7f46acfb2640 10 monclient: init 2023-07-29T14:05:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T14:05:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a80b74a0 0x7f46a80b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 10 monclient(hunting): picked mon.noname-a con 0x7f46a80b74a0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46a813dde0 0x7f46a80ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 10 monclient(hunting): picked mon.noname-b con 0x7f46a813dde0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f46a81445e0 0x7f46a8136240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 10 monclient(hunting): picked mon.noname-c con 0x7f46a81445e0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80ace80 con 0x7f46a80b74a0 2023-07-29T14:05:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80ac360 con 0x7f46a813dde0 2023-07-29T14:05:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80aca60 con 0x7f46a81445e0 2023-07-29T14:05:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46acfb2640 10 monclient(hunting): authenticate will time out at 3672.292526s 2023-07-29T14:05:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.160+0000 7f46a659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a80b74a0 0x7f46a80b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.161+0000 7f46a659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a80b74a0 0x7f46a80b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53908/0 (socket says 172.21.15.40:53908) 2023-07-29T14:05:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.161+0000 7f46a659c640 1 -- 172.21.15.40:0/2419209716 learned_addr learned my addr 172.21.15.40:0/2419209716 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:50.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.161+0000 7f46a659c640 10 monclient(hunting): get_auth_request con 0x7f46a80b74a0 auth_method 0 2023-07-29T14:05:50.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.161+0000 7f46a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:50.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.161+0000 7f46a659c640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:50.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.161+0000 7f46a659c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:50.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.161+0000 7f46a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:50.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.161+0000 7f46a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:50.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.161+0000 7f46a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:50.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.163+0000 7f46a6d9d640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f46a81445e0 0x7f46a8136240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.163+0000 7f46a6d9d640 10 monclient(hunting): get_auth_request con 0x7f46a81445e0 auth_method 0 2023-07-29T14:05:50.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.163+0000 7f46a6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:50.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.163+0000 7f46a6d9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:50.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.163+0000 7f46a6d9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:50.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.164+0000 7f46a6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:50.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.164+0000 7f46a6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.164+0000 7f46a6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.165+0000 7f46a659c640 10 monclient(hunting): handle_auth_done global_id 14943 payload 824 2023-07-29T14:05:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.165+0000 7f46a659c640 1 -- 172.21.15.40:0/2419209716 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f46a81445e0 msgr2=0x7f46a8136240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.165+0000 7f46a659c640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f46a81445e0 0x7f46a8136240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.165+0000 7f46a659c640 1 -- 172.21.15.40:0/2419209716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46a813dde0 msgr2=0x7f46a80ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.165+0000 7f46a659c640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46a813dde0 0x7f46a80ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.165+0000 7f46a659c640 10 monclient: _finish_hunting 0 2023-07-29T14:05:50.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.165+0000 7f46a659c640 1 monclient: found mon.noname-a 2023-07-29T14:05:50.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.165+0000 7f46a659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:50.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.165+0000 7f46a659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T14:05:50.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.165+0000 7f46a659c640 1 -- 172.21.15.40:0/2419209716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00093a0 con 0x7f46a80b74a0 2023-07-29T14:05:50.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.167+0000 7f46a659c640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a80b74a0 0x7f46a80b38b0 secure :-1 s=READY pgs=3217 cs=0 l=1 rev1=1 crypto rx=0x7f46a0000c00 tx=0x7f46a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:50.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.167+0000 7f469effd640 10 client.?.objecter ms_handle_connect 0x7f46a80b74a0 2023-07-29T14:05:50.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.167+0000 7f469effd640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:50.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 1 -- 172.21.15.40:0/2419209716 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0032810 con 0x7f46a80b74a0 2023-07-29T14:05:50.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:50.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T14:05:50.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 10 monclient: dump: 2023-07-29T14:05:50.198 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:50.198 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:50.200 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.201 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.201 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:50.201 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.202 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:50.202 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:50.202 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:50.202 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 10 monclient: _finish_auth 0 2023-07-29T14:05:50.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:50.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:50.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 1 -- 172.21.15.40:0/2419209716 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a0032970 con 0x7f46a80b74a0 2023-07-29T14:05:50.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:50.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 1 -- 172.21.15.40:0/2419209716 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0032d20 con 0x7f46a80b74a0 2023-07-29T14:05:50.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f46acfb2640 5 monclient: authenticate success, global_id 14943 2023-07-29T14:05:50.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f46acfb2640 10 monclient: _renew_subs 2023-07-29T14:05:50.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f46acfb2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:50.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f46acfb2640 1 -- 172.21.15.40:0/2419209716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a80b4150 con 0x7f46a80b74a0 2023-07-29T14:05:50.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f46a4d99640 4 set_mon_vals no callback set 2023-07-29T14:05:50.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:50.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:50.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f469effd640 10 monclient: dump: 2023-07-29T14:05:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:50.206 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.206 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:50.206 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:50.206 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:50.206 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f46acfb2640 10 client.14943.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:50.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f46acfb2640 10 monclient: _renew_subs 2023-07-29T14:05:50.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f46acfb2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:50.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f46acfb2640 1 -- 172.21.15.40:0/2419209716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a8136960 con 0x7f46a80b74a0 2023-07-29T14:05:50.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.172+0000 7f46acfb2640 1 librados: init done 2023-07-29T14:05:50.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.173+0000 7f46acfb2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.173+0000 7f46acfb2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.173+0000 7f46acfb2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.173+0000 7f46acfb2640 1 -- 172.21.15.40:0/2419209716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a804e000 con 0x7f46a80b74a0 2023-07-29T14:05:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.192+0000 7f469effd640 1 -- 172.21.15.40:0/2419209716 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f46a004e450 con 0x7f46a80b74a0 2023-07-29T14:05:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.193+0000 7f469effd640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f468806bfe0 0x7f468806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.193+0000 7f46a5d9b640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f468806bfe0 0x7f468806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.193+0000 7f46a5d9b640 10 monclient: get_auth_request con 0x7f468806bfe0 auth_method 0 2023-07-29T14:05:50.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.194+0000 7f469effd640 10 client.14943.objecter ms_handle_connect 0x7f468806bfe0 2023-07-29T14:05:50.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.194+0000 7f46a5d9b640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f468806bfe0 0x7f468806e490 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4690000f60 tx=0x7f4690007a80 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:50.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.194+0000 7f469effd640 1 -- 172.21.15.40:0/2419209716 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f46a007dfe0 con 0x7f46a80b74a0 2023-07-29T14:05:50.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.194+0000 7f469effd640 10 client.14943.objecter ms_dispatch 0x7f46a8000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:50.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.194+0000 7f469effd640 3 client.14943.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:50.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.194+0000 7f469effd640 3 client.14943.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:50.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.195+0000 7f469effd640 20 client.14943.objecter dump_active .. 0 homeless 2023-07-29T14:05:50.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.461+0000 7f469effd640 1 -- 172.21.15.40:0/2419209716 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f46a0076360 con 0x7f46a80b74a0 2023-07-29T14:05:50.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.461+0000 7f469effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:50.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.461+0000 7f469effd640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:50.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.641+0000 7f46acfb2640 1 -- 172.21.15.40:0/2419209716 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f46a80b12d0 con 0x7f468806bfe0 2023-07-29T14:05:50.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.642+0000 7f469effd640 1 -- 172.21.15.40:0/2419209716 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f46a80b12d0 con 0x7f468806bfe0 2023-07-29T14:05:50.642 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2023-07-29T14:05:50.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 10 librados: watch_flush enter 2023-07-29T14:05:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 10 librados: watch_flush exit 2023-07-29T14:05:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 20 client.14943.objecter shutdown clearing up homeless session... 2023-07-29T14:05:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 10 client.14943.objecter successfully canceled tick 2023-07-29T14:05:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 -- 172.21.15.40:0/2419209716 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f468806bfe0 msgr2=0x7f468806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f468806bfe0 0x7f468806e490 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4690000f60 tx=0x7f4690007a80 comp rx=0 tx=0).stop 2023-07-29T14:05:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 10 monclient: shutdown 2023-07-29T14:05:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 -- 172.21.15.40:0/2419209716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a80b74a0 msgr2=0x7f46a80b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a80b74a0 0x7f46a80b38b0 secure :-1 s=READY pgs=3217 cs=0 l=1 rev1=1 crypto rx=0x7f46a0000c00 tx=0x7f46a00046c0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 -- 172.21.15.40:0/2419209716 shutdown_connections 2023-07-29T14:05:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f468806bfe0 0x7f468806e490 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46a813dde0 0x7f46a80ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f46a81445e0 0x7f46a8136240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 --2- 172.21.15.40:0/2419209716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a80b74a0 0x7f46a80b38b0 unknown :-1 s=CLOSED pgs=3217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 -- 172.21.15.40:0/2419209716 >> 172.21.15.40:0/2419209716 conn(0x7f46a8065b00 msgr2=0x7f46a80b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 -- 172.21.15.40:0/2419209716 shutdown_connections 2023-07-29T14:05:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.644+0000 7f46acfb2640 1 -- 172.21.15.40:0/2419209716 wait complete. 2023-07-29T14:05:50.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.645+0000 7f46acfb2640 1 librados: shutdown 2023-07-29T14:05:50.659 INFO:tasks.ceph:Scrubbing osd.3 2023-07-29T14:05:50.659 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-07-29T14:05:50.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.770+0000 7f7fa8ed5640 10 monclient: get_monmap_and_config 2023-07-29T14:05:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.770+0000 7f7fa8ed5640 10 monclient: build_initial_monmap 2023-07-29T14:05:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.770+0000 7f7fa8ed5640 10 monclient: monmap: 2023-07-29T14:05:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:50.771163+0000 2023-07-29T14:05:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:50.771163+0000 2023-07-29T14:05:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:50.773 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.771+0000 7f7fa8ed5640 1 Processor -- start 2023-07-29T14:05:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.771+0000 7f7fa8ed5640 1 -- start start 2023-07-29T14:05:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.771+0000 7f7fa8ed5640 10 monclient: init 2023-07-29T14:05:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2023-07-29T14:05:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa40b3a60 0x7f7fa413de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 10 monclient(hunting): picked mon.noname-b con 0x7f7fa40b3a60 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7fa413e380 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fa4141330 0x7f7fa414b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 10 monclient(hunting): picked mon.noname-a con 0x7f7fa4141330 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa40b3e30 con 0x7f7fa413e380 2023-07-29T14:05:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa40b0e70 con 0x7f7fa4141330 2023-07-29T14:05:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:50.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40a8930 con 0x7f7fa40b3a60 2023-07-29T14:05:50.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:50.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa8ed5640 10 monclient(hunting): authenticate will time out at 3672.904488s 2023-07-29T14:05:50.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa259c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa40b3a60 0x7f7fa413de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.772+0000 7f7fa2d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fa4141330 0x7f7fa414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7f9bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7fa413e380 0x7f7fa4140c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.40:44958/0 (socket says 172.21.15.40:44958) 2023-07-29T14:05:50.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7f9bfff640 1 -- 172.21.15.40:0/2107787966 learned_addr learned my addr 172.21.15.40:0/2107787966 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:50.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7fa259c640 10 monclient(hunting): get_auth_request con 0x7f7fa40b3a60 auth_method 0 2023-07-29T14:05:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7fa259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7fa259c640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7fa259c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7fa2d9d640 10 monclient(hunting): get_auth_request con 0x7f7fa4141330 auth_method 0 2023-07-29T14:05:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7fa2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7fa2d9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7fa2d9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7fa259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:50.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7fa259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:50.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.773+0000 7f7fa259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:50.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.774+0000 7f7fa2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.774+0000 7f7fa2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.774+0000 7f7fa2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.775+0000 7f7fa259c640 10 monclient(hunting): handle_auth_done global_id 16391 payload 274 2023-07-29T14:05:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.775+0000 7f7fa259c640 1 -- 172.21.15.40:0/2107787966 >> v2:172.21.15.40:3300/0 conn(0x7f7fa4141330 msgr2=0x7f7fa414b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.775+0000 7f7fa259c640 1 --2- 172.21.15.40:0/2107787966 >> v2:172.21.15.40:3300/0 conn(0x7f7fa4141330 0x7f7fa414b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.775+0000 7f7fa259c640 1 -- 172.21.15.40:0/2107787966 >> v1:172.21.15.45:6789/0 conn(0x7f7fa413e380 legacy=0x7f7fa4140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.775+0000 7f7fa259c640 10 monclient: _finish_hunting 0 2023-07-29T14:05:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.775+0000 7f7fa259c640 1 monclient: found mon.noname-b 2023-07-29T14:05:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.775+0000 7f7fa259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.775+0000 7f7fa259c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.775+0000 7f7fa259c640 1 -- 172.21.15.40:0/2107787966 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40a9600 con 0x7f7fa40b3a60 2023-07-29T14:05:50.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.777+0000 7f7fa259c640 1 --2- 172.21.15.40:0/2107787966 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa40b3a60 0x7f7fa413de40 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7f7f940097a0 tx=0x7f7f9402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e688b8ac9d4a51 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:50.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 1 -- 172.21.15.40:0/2107787966 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9402e070 con 0x7f7fa40b3a60 2023-07-29T14:05:50.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:50.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:50.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 10 monclient: dump: 2023-07-29T14:05:50.787 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:50.787 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:50.787 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.787 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.787 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:50.788 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.788 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:50.788 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:50.788 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:50.788 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 10 monclient: _finish_auth 0 2023-07-29T14:05:50.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 10 monclient: _check_auth_tickets 2023-07-29T14:05:50.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:50.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 1 -- 172.21.15.40:0/2107787966 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f940315a0 con 0x7f7fa40b3a60 2023-07-29T14:05:50.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:50.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa8ed5640 5 monclient: authenticate success, global_id 16391 2023-07-29T14:05:50.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 1 -- 172.21.15.40:0/2107787966 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94031930 con 0x7f7fa40b3a60 2023-07-29T14:05:50.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:50.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:50.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa1d9b640 10 monclient: dump: 2023-07-29T14:05:50.790 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:50.790 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:50.790 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.791 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.791 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:50.791 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.791 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:50.791 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:50.791 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:50.792 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa8ed5640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.781+0000 7f7fa8ed5640 4 set_mon_vals no callback set 2023-07-29T14:05:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.782+0000 7f7fa8ed5640 10 monclient: shutdown 2023-07-29T14:05:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.782+0000 7f7fa8ed5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.782+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/2107787966 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa40b3a60 msgr2=0x7f7fa413de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.782+0000 7f7fa8ed5640 1 --2- 172.21.15.40:0/2107787966 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa40b3a60 0x7f7fa413de40 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7f7f940097a0 tx=0x7f7f9402b840 comp rx=0 tx=0).stop 2023-07-29T14:05:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.782+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/2107787966 shutdown_connections 2023-07-29T14:05:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.782+0000 7f7fa8ed5640 1 --2- 172.21.15.40:0/2107787966 >> v2:172.21.15.40:3300/0 conn(0x7f7fa4141330 0x7f7fa414b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.782+0000 7f7fa8ed5640 1 --2- 172.21.15.40:0/2107787966 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa40b3a60 0x7f7fa413de40 unknown :-1 s=CLOSED pgs=3228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.782+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/2107787966 >> 172.21.15.40:0/2107787966 conn(0x7f7fa4065920 msgr2=0x7f7fa4065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.782+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/2107787966 shutdown_connections 2023-07-29T14:05:50.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.782+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/2107787966 wait complete. 2023-07-29T14:05:50.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.783+0000 7f7fa8ed5640 10 monclient: build_initial_monmap 2023-07-29T14:05:50.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.783+0000 7f7fa8ed5640 10 monclient: monmap: 2023-07-29T14:05:50.794 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:50.794 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:50.794 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:50.794 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:50.794 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:50.795 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.795 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:50.795 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:50.795 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:50.795 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.783+0000 7f7fa8ed5640 1 librados: starting msgr at 2023-07-29T14:05:50.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.783+0000 7f7fa8ed5640 1 librados: starting objecter 2023-07-29T14:05:50.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.783+0000 7f7fa8ed5640 1 Processor -- start 2023-07-29T14:05:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.783+0000 7f7fa8ed5640 1 -- start start 2023-07-29T14:05:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.783+0000 7f7fa8ed5640 1 librados: setting wanted keys 2023-07-29T14:05:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.783+0000 7f7fa8ed5640 1 librados: calling monclient init 2023-07-29T14:05:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.783+0000 7f7fa8ed5640 10 monclient: init 2023-07-29T14:05:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T14:05:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fa40b3a60 0x7f7fa4144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 10 monclient(hunting): picked mon.noname-b con 0x7f7fa40b3a60 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa413e380 0x7f7fa40ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 10 monclient(hunting): picked mon.noname-c con 0x7f7fa413e380 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fa4141330 0x7f7fa4139010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 10 monclient(hunting): picked mon.noname-a con 0x7f7fa4141330 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4144ee0 con 0x7f7fa4141330 2023-07-29T14:05:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40ac970 con 0x7f7fa40b3a60 2023-07-29T14:05:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40ab8b0 con 0x7f7fa413e380 2023-07-29T14:05:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7fa8ed5640 10 monclient(hunting): authenticate will time out at 3672.916499s 2023-07-29T14:05:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.784+0000 7f7f9bfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa413e380 0x7f7fa40ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fa4141330 0x7f7fa4139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7f9bfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa413e380 0x7f7fa40ac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56622/0 (socket says 172.21.15.40:56622) 2023-07-29T14:05:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7f9bfff640 1 -- 172.21.15.40:0/1513818572 learned_addr learned my addr 172.21.15.40:0/1513818572 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7f9bfff640 10 monclient(hunting): get_auth_request con 0x7f7fa413e380 auth_method 0 2023-07-29T14:05:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:50.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7f9bfff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:50.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7f9bfff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:50.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa259c640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fa40b3a60 0x7f7fa4144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa2d9d640 10 monclient(hunting): get_auth_request con 0x7f7fa4141330 auth_method 0 2023-07-29T14:05:50.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:50.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa2d9d640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:50.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa2d9d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:50.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa259c640 10 monclient(hunting): get_auth_request con 0x7f7fa40b3a60 auth_method 0 2023-07-29T14:05:50.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:50.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa259c640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:50.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa259c640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:50.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7f9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:50.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7f9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:50.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7f9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:50.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:50.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:50.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.785+0000 7f7fa2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:50.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.786+0000 7f7fa259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:50.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.786+0000 7f7fa259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.786+0000 7f7fa259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:50.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.788+0000 7f7f9bfff640 10 monclient(hunting): handle_auth_done global_id 16394 payload 824 2023-07-29T14:05:50.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.788+0000 7f7f9bfff640 1 -- 172.21.15.40:0/1513818572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fa40b3a60 msgr2=0x7f7fa4144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.788+0000 7f7f9bfff640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fa40b3a60 0x7f7fa4144100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.788+0000 7f7f9bfff640 1 -- 172.21.15.40:0/1513818572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fa4141330 msgr2=0x7f7fa4139010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.788+0000 7f7f9bfff640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fa4141330 0x7f7fa4139010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.788+0000 7f7f9bfff640 10 monclient: _finish_hunting 0 2023-07-29T14:05:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.788+0000 7f7f9bfff640 1 monclient: found mon.noname-c 2023-07-29T14:05:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.788+0000 7f7f9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.788+0000 7f7f9bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.788+0000 7f7f9bfff640 1 -- 172.21.15.40:0/1513818572 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f94009400 con 0x7f7fa413e380 2023-07-29T14:05:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.791+0000 7f7f9bfff640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa413e380 0x7f7fa40ac350 secure :-1 s=READY pgs=3229 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c4a0 tx=0x7f7f8c00c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:50.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.791+0000 7f7f9affd640 10 client.?.objecter ms_handle_connect 0x7f7fa413e380 2023-07-29T14:05:50.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.791+0000 7f7f9affd640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:50.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 1 -- 172.21.15.40:0/1513818572 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c005ce0 con 0x7f7fa413e380 2023-07-29T14:05:50.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:50.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:50.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 10 monclient: dump: 2023-07-29T14:05:50.806 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:50.806 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:50.807 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.807 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.807 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:50.807 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.807 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:50.807 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:50.807 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:50.808 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 10 monclient: _finish_auth 0 2023-07-29T14:05:50.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:50.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:50.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 1 -- 172.21.15.40:0/1513818572 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c005e40 con 0x7f7fa413e380 2023-07-29T14:05:50.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7fa8ed5640 5 monclient: authenticate success, global_id 16394 2023-07-29T14:05:50.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:50.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 1 -- 172.21.15.40:0/1513818572 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c007270 con 0x7f7fa413e380 2023-07-29T14:05:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7f9affd640 10 monclient: dump: 2023-07-29T14:05:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:50.810 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:50.810 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:50.810 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:50.810 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:50.811 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7fa159a640 4 set_mon_vals no callback set 2023-07-29T14:05:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7fa8ed5640 10 monclient: _renew_subs 2023-07-29T14:05:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7fa8ed5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/1513818572 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa40abb30 con 0x7f7fa413e380 2023-07-29T14:05:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7fa8ed5640 10 client.16394.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7fa8ed5640 10 monclient: _renew_subs 2023-07-29T14:05:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7fa8ed5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/1513818572 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa4139730 con 0x7f7fa413e380 2023-07-29T14:05:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.794+0000 7f7fa8ed5640 1 librados: init done 2023-07-29T14:05:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.795+0000 7f7fa8ed5640 10 client.16394.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.795+0000 7f7fa8ed5640 15 client.16394.objecter _session_command_op_assign -1 1 2023-07-29T14:05:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.795+0000 7f7fa8ed5640 20 client.16394.objecter _get_session osd=-1 returning homeless 2023-07-29T14:05:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.795+0000 7f7fa8ed5640 10 client.16394.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.795+0000 7f7fa8ed5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.795+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/1513818572 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7fa40a9bc0 con 0x7f7fa413e380 2023-07-29T14:05:50.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.816+0000 7f7f9affd640 1 -- 172.21.15.40:0/1513818572 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f7f8c007490 con 0x7f7fa413e380 2023-07-29T14:05:50.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.816+0000 7f7f9affd640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7f8406bfb0 0x7f7f8406e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.817+0000 7f7fa259c640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7f8406bfb0 0x7f7f8406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.817+0000 7f7fa259c640 10 monclient: get_auth_request con 0x7f7f8406bfb0 auth_method 0 2023-07-29T14:05:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.817+0000 7f7f9affd640 10 client.16394.objecter ms_handle_connect 0x7f7f8406bfb0 2023-07-29T14:05:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.817+0000 7f7fa259c640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7f8406bfb0 0x7f7f8406e460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7f94036000 tx=0x7f7f9403d040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 1 -- 172.21.15.40:0/1513818572 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f7f8c05b830 con 0x7f7fa413e380 2023-07-29T14:05:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 10 client.16394.objecter ms_dispatch 0x7f7fa4000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 3 client.16394.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 3 client.16394.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 10 client.16394.objecter checking command 1 2023-07-29T14:05:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] conn(0x7f7f840737f0 0x7f7f84075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 20 client.16394.objecter _get_session s=0x7f7f84073120 osd=3 3 2023-07-29T14:05:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 20 client.16394.objecter put_session s=0x7f7f84073120 osd=3 3 2023-07-29T14:05:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 15 client.16394.objecter _session_command_op_remove -1 1 2023-07-29T14:05:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 20 client.16394.objecter _get_session s=0x7f7f84073120 osd=3 3 2023-07-29T14:05:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 20 client.16394.objecter get_session s=0x7f7f84073120 osd=3 3 2023-07-29T14:05:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 15 client.16394.objecter _session_command_op_assign 3 1 2023-07-29T14:05:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 20 client.16394.objecter put_session s=0x7f7f84073120 osd=3 4 2023-07-29T14:05:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 10 client.16394.objecter _send_command 1 2023-07-29T14:05:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 1 -- 172.21.15.40:0/1513818572 --> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f84076350 con 0x7f7f840737f0 2023-07-29T14:05:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 20 client.16394.objecter dump_active .. 0 homeless 2023-07-29T14:05:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 1 -- 172.21.15.40:0/1513818572 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=1004) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f8c053c60 con 0x7f7fa413e380 2023-07-29T14:05:50.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.818+0000 7f7f9affd640 10 monclient: handle_get_version_reply finishing 1 version 1004 2023-07-29T14:05:50.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.819+0000 7f7fa2d9d640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] conn(0x7f7f840737f0 0x7f7f84075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:50.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.819+0000 7f7fa2d9d640 10 monclient: get_auth_request con 0x7f7f840737f0 auth_method 0 2023-07-29T14:05:50.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.822+0000 7f7fa2d9d640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] conn(0x7f7f840737f0 0x7f7f84075c80 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:50.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.822+0000 7f7f9affd640 10 client.16394.objecter ms_handle_connect 0x7f7f840737f0 2023-07-29T14:05:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.845+0000 7f7f9affd640 1 -- 172.21.15.40:0/1513818572 <== osd.3 v2:172.21.15.40:6824/85571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7f84076350 con 0x7f7f840737f0 2023-07-29T14:05:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.845+0000 7f7f9affd640 10 client.16394.objecter ms_dispatch 0x7f7fa4000b90 command_reply(tid 1: 0 ) v1 2023-07-29T14:05:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.845+0000 7f7f9affd640 10 client.16394.objecter _finish_command 1 = system:0 2023-07-29T14:05:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.845+0000 7f7f9affd640 20 client.16394.objecter put_session s=0x7f7f84073120 osd=3 4 2023-07-29T14:05:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.845+0000 7f7f9affd640 15 client.16394.objecter _session_command_op_remove 3 1 2023-07-29T14:05:50.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.858+0000 7f7fa8ed5640 10 client.16394.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-07-29T14:05:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.858+0000 7f7fa8ed5640 15 client.16394.objecter _session_command_op_assign -1 2 2023-07-29T14:05:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.858+0000 7f7fa8ed5640 20 client.16394.objecter _get_session s=0x7f7f84073120 osd=3 3 2023-07-29T14:05:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.858+0000 7f7fa8ed5640 20 client.16394.objecter put_session s=0x7f7f84073120 osd=3 3 2023-07-29T14:05:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.858+0000 7f7fa8ed5640 20 client.16394.objecter _get_session s=0x7f7f84073120 osd=3 3 2023-07-29T14:05:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.858+0000 7f7fa8ed5640 15 client.16394.objecter _session_command_op_remove -1 2 2023-07-29T14:05:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.858+0000 7f7fa8ed5640 20 client.16394.objecter get_session s=0x7f7f84073120 osd=3 3 2023-07-29T14:05:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.858+0000 7f7fa8ed5640 15 client.16394.objecter _session_command_op_assign 3 2 2023-07-29T14:05:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.858+0000 7f7fa8ed5640 20 client.16394.objecter put_session s=0x7f7f84073120 osd=3 4 2023-07-29T14:05:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.858+0000 7f7fa8ed5640 10 client.16394.objecter _send_command 2 2023-07-29T14:05:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.858+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/1513818572 --> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7fa40a98e0 con 0x7f7f840737f0 2023-07-29T14:05:50.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.872+0000 7f7f9affd640 1 -- 172.21.15.40:0/1513818572 <== osd.3 v2:172.21.15.40:6824/85571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7f9c00f040 con 0x7f7f840737f0 2023-07-29T14:05:50.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.872+0000 7f7f9affd640 10 client.16394.objecter ms_dispatch 0x7f7fa4000b90 command_reply(tid 2: 0 ) v1 2023-07-29T14:05:50.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.872+0000 7f7f9affd640 10 client.16394.objecter _finish_command 2 = system:0 2023-07-29T14:05:50.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.872+0000 7f7f9affd640 20 client.16394.objecter put_session s=0x7f7f84073120 osd=3 4 2023-07-29T14:05:50.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.872+0000 7f7f9affd640 15 client.16394.objecter _session_command_op_remove 3 2 2023-07-29T14:05:50.878 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-29T14:05:50.880 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-29T14:05:50.880 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-29T14:05:50.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 10 librados: watch_flush enter 2023-07-29T14:05:50.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 10 librados: watch_flush exit 2023-07-29T14:05:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 10 client.16394.objecter close_session for osd.3 2023-07-29T14:05:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/1513818572 >> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] conn(0x7f7f840737f0 msgr2=0x7f7f84075c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.40:6824/85571,v1:172.21.15.40:6825/85571] conn(0x7f7f840737f0 0x7f7f84075c80 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 20 client.16394.objecter put_session s=0x7f7f84073120 osd=3 1 2023-07-29T14:05:50.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 20 client.16394.objecter shutdown clearing up homeless session... 2023-07-29T14:05:50.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 10 client.16394.objecter successfully canceled tick 2023-07-29T14:05:50.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/1513818572 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7f8406bfb0 msgr2=0x7f7f8406e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f7f8406bfb0 0x7f7f8406e460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7f94036000 tx=0x7f7f9403d040 comp rx=0 tx=0).stop 2023-07-29T14:05:50.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 10 monclient: shutdown 2023-07-29T14:05:50.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:50.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/1513818572 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa413e380 msgr2=0x7f7fa40ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:50.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 1 --2- 172.21.15.40:0/1513818572 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7fa413e380 0x7f7fa40ac350 secure :-1 s=READY pgs=3229 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c4a0 tx=0x7f7f8c00c960 comp rx=0 tx=0).stop 2023-07-29T14:05:50.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa2d9d640 1 -- 172.21.15.40:0/1513818572 reap_dead start 2023-07-29T14:05:50.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/1513818572 shutdown_connections 2023-07-29T14:05:50.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/1513818572 >> 172.21.15.40:0/1513818572 conn(0x7f7fa4065920 msgr2=0x7f7fa413c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:50.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/1513818572 shutdown_connections 2023-07-29T14:05:50.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 1 -- 172.21.15.40:0/1513818572 wait complete. 2023-07-29T14:05:50.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.873+0000 7f7fa8ed5640 1 librados: shutdown 2023-07-29T14:05:50.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-07-29T14:05:50.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.998+0000 7facab057640 10 monclient: get_monmap_and_config 2023-07-29T14:05:51.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.998+0000 7facab057640 10 monclient: build_initial_monmap 2023-07-29T14:05:51.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.998+0000 7facab057640 10 monclient: monmap: 2023-07-29T14:05:51.000 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:51.000 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.000 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:50.999009+0000 2023-07-29T14:05:51.000 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:50.999009+0000 2023-07-29T14:05:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:51.001 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.999+0000 7facab057640 1 Processor -- start 2023-07-29T14:05:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.999+0000 7facab057640 1 -- start start 2023-07-29T14:05:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:50.999+0000 7facab057640 10 monclient: init 2023-07-29T14:05:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2023-07-29T14:05:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca40ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 10 monclient(hunting): picked mon.noname-b con 0x7faca40af800 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faca40afbd0 0x7faca413e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 10 monclient(hunting): picked mon.noname-a con 0x7faca40afbd0 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faca40aca90 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca4071410 con 0x7faca40aca90 2023-07-29T14:05:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca40ac390 con 0x7faca40afbd0 2023-07-29T14:05:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca40ac670 con 0x7faca40af800 2023-07-29T14:05:51.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:51.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7facab057640 10 monclient(hunting): authenticate will time out at 3673.132277s 2023-07-29T14:05:51.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7faca8df3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca40ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.000+0000 7faca3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faca40afbd0 0x7faca413e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca8df3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca40ab160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56632/0 (socket says 172.21.15.40:56632) 2023-07-29T14:05:51.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca8df3640 1 -- 172.21.15.40:0/3782547025 learned_addr learned my addr 172.21.15.40:0/3782547025 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:51.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca8df3640 10 monclient(hunting): get_auth_request con 0x7faca40af800 auth_method 0 2023-07-29T14:05:51.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca8df3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca8df3640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca8df3640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca3fff640 10 monclient(hunting): get_auth_request con 0x7faca40afbd0 auth_method 0 2023-07-29T14:05:51.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca3fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca3fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca8df3640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca8df3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca8df3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.001+0000 7faca3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.003+0000 7faca8df3640 10 monclient(hunting): handle_auth_done global_id 16397 payload 274 2023-07-29T14:05:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.003+0000 7faca8df3640 1 -- 172.21.15.40:0/3782547025 >> v2:172.21.15.40:3300/0 conn(0x7faca40afbd0 msgr2=0x7faca413e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.003+0000 7faca8df3640 1 --2- 172.21.15.40:0/3782547025 >> v2:172.21.15.40:3300/0 conn(0x7faca40afbd0 0x7faca413e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.003+0000 7faca8df3640 1 -- 172.21.15.40:0/3782547025 >> v1:172.21.15.45:6789/0 conn(0x7faca40aca90 legacy=0x7faca4070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.003+0000 7faca8df3640 10 monclient: _finish_hunting 0 2023-07-29T14:05:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.003+0000 7faca8df3640 1 monclient: found mon.noname-b 2023-07-29T14:05:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.003+0000 7faca8df3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.003+0000 7faca8df3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.003+0000 7faca8df3640 1 -- 172.21.15.40:0/3782547025 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca40aac40 con 0x7faca40af800 2023-07-29T14:05:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.005+0000 7faca8df3640 1 --2- 172.21.15.40:0/3782547025 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca40ab160 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 crypto rx=0x7fac98009870 tx=0x7fac9802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668511013a2c1eb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.007+0000 7faca37fe640 1 -- 172.21.15.40:0/3782547025 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98030840 con 0x7faca40af800 2023-07-29T14:05:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.007+0000 7faca37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.007+0000 7faca37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.007+0000 7faca37fe640 10 monclient: dump: 2023-07-29T14:05:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.014 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.007+0000 7faca37fe640 10 monclient: _finish_auth 0 2023-07-29T14:05:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.007+0000 7faca37fe640 10 monclient: _check_auth_tickets 2023-07-29T14:05:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.007+0000 7faca37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.007+0000 7facab057640 5 monclient: authenticate success, global_id 16397 2023-07-29T14:05:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.007+0000 7facab057640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T14:05:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.009+0000 7faca37fe640 1 -- 172.21.15.40:0/3782547025 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac980309a0 con 0x7faca40af800 2023-07-29T14:05:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.009+0000 7faca37fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.009+0000 7faca37fe640 1 -- 172.21.15.40:0/3782547025 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98030d30 con 0x7faca40af800 2023-07-29T14:05:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.009+0000 7faca37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.009+0000 7faca37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.009+0000 7faca37fe640 10 monclient: dump: 2023-07-29T14:05:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.016 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.016 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.016 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.017 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.017 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.017 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.017 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.017 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.017 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.009+0000 7facab057640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:51.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.009+0000 7facab057640 4 set_mon_vals no callback set 2023-07-29T14:05:51.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.010+0000 7facab057640 10 monclient: shutdown 2023-07-29T14:05:51.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.010+0000 7facab057640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:51.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.010+0000 7facab057640 1 -- 172.21.15.40:0/3782547025 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 msgr2=0x7faca40ab160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.010+0000 7facab057640 1 --2- 172.21.15.40:0/3782547025 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca40ab160 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 crypto rx=0x7fac98009870 tx=0x7fac9802b980 comp rx=0 tx=0).stop 2023-07-29T14:05:51.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.010+0000 7facab057640 1 -- 172.21.15.40:0/3782547025 shutdown_connections 2023-07-29T14:05:51.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.010+0000 7facab057640 1 --2- 172.21.15.40:0/3782547025 >> v2:172.21.15.40:3300/0 conn(0x7faca40afbd0 0x7faca413e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.010+0000 7facab057640 1 --2- 172.21.15.40:0/3782547025 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca40ab160 unknown :-1 s=CLOSED pgs=3230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.010+0000 7facab057640 1 -- 172.21.15.40:0/3782547025 >> 172.21.15.40:0/3782547025 conn(0x7faca4065b00 msgr2=0x7faca4065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:51.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.010+0000 7facab057640 1 -- 172.21.15.40:0/3782547025 shutdown_connections 2023-07-29T14:05:51.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.010+0000 7facab057640 1 -- 172.21.15.40:0/3782547025 wait complete. 2023-07-29T14:05:51.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.011+0000 7facab057640 10 monclient: build_initial_monmap 2023-07-29T14:05:51.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.011+0000 7facab057640 10 monclient: monmap: 2023-07-29T14:05:51.020 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:51.022 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.011+0000 7facab057640 1 librados: starting msgr at 2023-07-29T14:05:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.011+0000 7facab057640 1 librados: starting objecter 2023-07-29T14:05:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.011+0000 7facab057640 1 Processor -- start 2023-07-29T14:05:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.011+0000 7facab057640 1 -- start start 2023-07-29T14:05:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.011+0000 7facab057640 1 librados: setting wanted keys 2023-07-29T14:05:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.011+0000 7facab057640 1 librados: calling monclient init 2023-07-29T14:05:51.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.011+0000 7facab057640 10 monclient: init 2023-07-29T14:05:51.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:51.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T14:05:51.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faca40aca90 0x7faca4141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 10 monclient(hunting): picked mon.noname-a con 0x7faca40aca90 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca413aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 10 monclient(hunting): picked mon.noname-c con 0x7faca40af800 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40afbd0 0x7faca4135f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 10 monclient(hunting): picked mon.noname-b con 0x7faca40afbd0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca413b300 con 0x7faca40aca90 2023-07-29T14:05:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca413af90 con 0x7faca40afbd0 2023-07-29T14:05:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca41427f0 con 0x7faca40af800 2023-07-29T14:05:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.012+0000 7facab057640 10 monclient(hunting): authenticate will time out at 3673.144772s 2023-07-29T14:05:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca3fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca413aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca8df3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faca40aca90 0x7faca4141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca95f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40afbd0 0x7faca4135f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca3fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca413aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56644/0 (socket says 172.21.15.40:56644) 2023-07-29T14:05:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca3fff640 1 -- 172.21.15.40:0/2740561600 learned_addr learned my addr 172.21.15.40:0/2740561600 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca3fff640 10 monclient(hunting): get_auth_request con 0x7faca40af800 auth_method 0 2023-07-29T14:05:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca3fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca3fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca8df3640 10 monclient(hunting): get_auth_request con 0x7faca40aca90 auth_method 0 2023-07-29T14:05:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca8df3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca8df3640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca8df3640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca95f4640 10 monclient(hunting): get_auth_request con 0x7faca40afbd0 auth_method 0 2023-07-29T14:05:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca95f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca95f4640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.013+0000 7faca95f4640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.014+0000 7faca3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.014+0000 7faca3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.014+0000 7faca3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.014+0000 7faca95f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.014+0000 7faca95f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.014+0000 7faca95f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.014+0000 7faca8df3640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.014+0000 7faca8df3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.014+0000 7faca8df3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.017+0000 7faca3fff640 10 monclient(hunting): handle_auth_done global_id 16400 payload 824 2023-07-29T14:05:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.017+0000 7faca3fff640 1 -- 172.21.15.40:0/2740561600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40afbd0 msgr2=0x7faca4135f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.017+0000 7faca3fff640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40afbd0 0x7faca4135f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.017+0000 7faca3fff640 1 -- 172.21.15.40:0/2740561600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faca40aca90 msgr2=0x7faca4141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.017+0000 7faca3fff640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faca40aca90 0x7faca4141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.017+0000 7faca3fff640 10 monclient: _finish_hunting 0 2023-07-29T14:05:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.017+0000 7faca3fff640 1 monclient: found mon.noname-c 2023-07-29T14:05:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.017+0000 7faca3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.017+0000 7faca3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.017+0000 7faca3fff640 1 -- 172.21.15.40:0/2740561600 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980094d0 con 0x7faca40af800 2023-07-29T14:05:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.019+0000 7faca3fff640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca413aa50 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 crypto rx=0x7fac9000e870 tx=0x7fac9000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.019+0000 7faca17fa640 10 client.?.objecter ms_handle_connect 0x7faca40af800 2023-07-29T14:05:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.019+0000 7faca17fa640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 1 -- 172.21.15.40:0/2740561600 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac900094f0 con 0x7faca40af800 2023-07-29T14:05:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 10 monclient: dump: 2023-07-29T14:05:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.037 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 10 monclient: _finish_auth 0 2023-07-29T14:05:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 10 monclient: _check_auth_tickets 2023-07-29T14:05:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 1 -- 172.21.15.40:0/2740561600 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9000c940 con 0x7faca40af800 2023-07-29T14:05:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7facab057640 5 monclient: authenticate success, global_id 16400 2023-07-29T14:05:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7facab057640 10 monclient: _renew_subs 2023-07-29T14:05:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7facab057640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7facab057640 1 -- 172.21.15.40:0/2740561600 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca413b610 con 0x7faca40af800 2023-07-29T14:05:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 1 -- 172.21.15.40:0/2740561600 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000ccd0 con 0x7faca40af800 2023-07-29T14:05:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca17fa640 10 monclient: dump: 2023-07-29T14:05:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.040 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.040 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.040 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.041 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7faca2ffd640 4 set_mon_vals no callback set 2023-07-29T14:05:51.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7facab057640 10 client.16400.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7facab057640 10 monclient: _renew_subs 2023-07-29T14:05:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7facab057640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7facab057640 1 -- 172.21.15.40:0/2740561600 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca4136650 con 0x7faca40af800 2023-07-29T14:05:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.023+0000 7facab057640 1 librados: init done 2023-07-29T14:05:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.024+0000 7facab057640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.024+0000 7facab057640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.024+0000 7facab057640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.024+0000 7facab057640 1 -- 172.21.15.40:0/2740561600 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca404e000 con 0x7faca40af800 2023-07-29T14:05:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.043+0000 7faca17fa640 1 -- 172.21.15.40:0/2740561600 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fac90013470 con 0x7faca40af800 2023-07-29T14:05:51.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.044+0000 7faca17fa640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fac8806bf60 0x7fac8806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.044+0000 7faca8df3640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fac8806bf60 0x7fac8806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.044+0000 7faca8df3640 10 monclient: get_auth_request con 0x7fac8806bf60 auth_method 0 2023-07-29T14:05:51.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.044+0000 7faca8df3640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fac8806bf60 0x7fac8806e410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fac98036000 tx=0x7fac98008030 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:51.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.044+0000 7faca17fa640 10 client.16400.objecter ms_handle_connect 0x7fac8806bf60 2023-07-29T14:05:51.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.047+0000 7faca17fa640 1 -- 172.21.15.40:0/2740561600 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7fac9005c3b0 con 0x7faca40af800 2023-07-29T14:05:51.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.047+0000 7faca17fa640 10 client.16400.objecter ms_dispatch 0x7faca4000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:51.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.047+0000 7faca17fa640 3 client.16400.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:51.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.047+0000 7faca17fa640 3 client.16400.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:51.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.047+0000 7faca17fa640 20 client.16400.objecter dump_active .. 0 homeless 2023-07-29T14:05:51.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.310+0000 7faca17fa640 1 -- 172.21.15.40:0/2740561600 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7fac900547e0 con 0x7faca40af800 2023-07-29T14:05:51.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.310+0000 7faca17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:51.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.310+0000 7faca17fa640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:51.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.488+0000 7facab057640 1 -- 172.21.15.40:0/2740561600 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7faca413c470 con 0x7fac8806bf60 2023-07-29T14:05:51.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.488+0000 7faca17fa640 1 -- 172.21.15.40:0/2740561600 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fac98030640 con 0x7fac8806bf60 2023-07-29T14:05:51.489 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2023-07-29T14:05:51.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 10 librados: watch_flush enter 2023-07-29T14:05:51.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 10 librados: watch_flush exit 2023-07-29T14:05:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 20 client.16400.objecter shutdown clearing up homeless session... 2023-07-29T14:05:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 10 client.16400.objecter successfully canceled tick 2023-07-29T14:05:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 1 -- 172.21.15.40:0/2740561600 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fac8806bf60 msgr2=0x7fac8806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fac8806bf60 0x7fac8806e410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fac98036000 tx=0x7fac98008030 comp rx=0 tx=0).stop 2023-07-29T14:05:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 10 monclient: shutdown 2023-07-29T14:05:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:51.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 1 -- 172.21.15.40:0/2740561600 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 msgr2=0x7faca413aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca413aa50 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 crypto rx=0x7fac9000e870 tx=0x7fac9000ed30 comp rx=0 tx=0).stop 2023-07-29T14:05:51.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 1 -- 172.21.15.40:0/2740561600 shutdown_connections 2023-07-29T14:05:51.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fac8806bf60 0x7fac8806e410 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40afbd0 0x7faca4135f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7faca40af800 0x7faca413aa50 unknown :-1 s=CLOSED pgs=3231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 1 --2- 172.21.15.40:0/2740561600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faca40aca90 0x7faca4141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.490+0000 7facab057640 1 -- 172.21.15.40:0/2740561600 >> 172.21.15.40:0/2740561600 conn(0x7faca4065b00 msgr2=0x7faca4065ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:51.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.491+0000 7facab057640 1 -- 172.21.15.40:0/2740561600 shutdown_connections 2023-07-29T14:05:51.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.491+0000 7facab057640 1 -- 172.21.15.40:0/2740561600 wait complete. 2023-07-29T14:05:51.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.491+0000 7facab057640 1 librados: shutdown 2023-07-29T14:05:51.505 INFO:tasks.ceph:Scrubbing osd.4 2023-07-29T14:05:51.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-07-29T14:05:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.619+0000 7f4f106d5640 10 monclient: get_monmap_and_config 2023-07-29T14:05:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.619+0000 7f4f106d5640 10 monclient: build_initial_monmap 2023-07-29T14:05:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.619+0000 7f4f106d5640 10 monclient: monmap: 2023-07-29T14:05:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:51.619797+0000 2023-07-29T14:05:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:51.619797+0000 2023-07-29T14:05:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:51.621 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.620+0000 7f4f106d5640 1 Processor -- start 2023-07-29T14:05:51.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.620+0000 7f4f106d5640 1 -- start start 2023-07-29T14:05:51.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.620+0000 7f4f106d5640 10 monclient: init 2023-07-29T14:05:51.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:51.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 10 monclient: _add_conns ranks=[0,3,4,2,1] 2023-07-29T14:05:51.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4f080b3a60 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:51.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4f08140c20 0x7f4f08141080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 10 monclient(hunting): picked mon.noname-a con 0x7f4f08140c20 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:51.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4f08149520 0x7f4f0814b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 10 monclient(hunting): picked mon.noname-c con 0x7f4f08149520 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:51.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f080b3e30 con 0x7f4f080b3a60 2023-07-29T14:05:51.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f080b0e70 con 0x7f4f08140c20 2023-07-29T14:05:51.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f080a8930 con 0x7f4f08149520 2023-07-29T14:05:51.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:51.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f106d5640 10 monclient(hunting): authenticate will time out at 3673.753259s 2023-07-29T14:05:51.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.621+0000 7f4f0dc70640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4f08140c20 0x7f4f08141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.627+0000 7f4f0ec72640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4f08149520 0x7f4f0814b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.676+0000 7f4f0e471640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4f080b3a60 0x7f4f0813de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38640/0 (socket says 172.21.15.40:38640) 2023-07-29T14:05:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.676+0000 7f4f0e471640 1 -- 172.21.15.40:0/3496382980 learned_addr learned my addr 172.21.15.40:0/3496382980 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.677+0000 7f4f0ec72640 10 monclient(hunting): get_auth_request con 0x7f4f08149520 auth_method 0 2023-07-29T14:05:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.677+0000 7f4f0ec72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.677+0000 7f4f0ec72640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.677+0000 7f4f0ec72640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.678+0000 7f4f0dc70640 10 monclient(hunting): get_auth_request con 0x7f4f08140c20 auth_method 0 2023-07-29T14:05:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.678+0000 7f4f0dc70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.678+0000 7f4f0dc70640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.678+0000 7f4f0dc70640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.680+0000 7f4f0ec72640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.680+0000 7f4f0ec72640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.680+0000 7f4f0ec72640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.681+0000 7f4f0dc70640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.681+0000 7f4f0dc70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.681+0000 7f4f0dc70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.682+0000 7f4f0ec72640 10 monclient(hunting): handle_auth_done global_id 20386 payload 274 2023-07-29T14:05:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.682+0000 7f4f0ec72640 1 -- 172.21.15.40:0/3496382980 >> v2:172.21.15.40:3300/0 conn(0x7f4f08140c20 msgr2=0x7f4f08141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.682+0000 7f4f0ec72640 1 --2- 172.21.15.40:0/3496382980 >> v2:172.21.15.40:3300/0 conn(0x7f4f08140c20 0x7f4f08141080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.682+0000 7f4f0ec72640 1 -- 172.21.15.40:0/3496382980 >> v1:172.21.15.40:6789/0 conn(0x7f4f080b3a60 legacy=0x7f4f0813de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.682+0000 7f4f0ec72640 10 monclient: _finish_hunting 0 2023-07-29T14:05:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.682+0000 7f4f0ec72640 1 monclient: found mon.noname-c 2023-07-29T14:05:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.682+0000 7f4f0ec72640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.682+0000 7f4f0ec72640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T14:05:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.682+0000 7f4f0ec72640 1 -- 172.21.15.40:0/3496382980 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f081415c0 con 0x7f4f08149520 2023-07-29T14:05:51.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.692+0000 7f4f0ec72640 1 --2- 172.21.15.40:0/3496382980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f08149520 0x7f4f0814b900 secure :-1 s=READY pgs=4786 cs=0 l=1 rev1=1 crypto rx=0x7f4f08070a10 tx=0x7f4f0002be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70347d884b6ade4c server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 1 -- 172.21.15.40:0/3496382980 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0002cbf0 con 0x7f4f08149520 2023-07-29T14:05:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:51.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 10 monclient: dump: 2023-07-29T14:05:51.699 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.699 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.700 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.700 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.700 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.700 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.701 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 10 monclient: _finish_auth 0 2023-07-29T14:05:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 10 monclient: _check_auth_tickets 2023-07-29T14:05:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 1 -- 172.21.15.40:0/3496382980 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0002cd50 con 0x7f4f08149520 2023-07-29T14:05:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 1 -- 172.21.15.40:0/3496382980 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f00034660 con 0x7f4f08149520 2023-07-29T14:05:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T14:05:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f0d46f640 10 monclient: dump: 2023-07-29T14:05:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.707 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f106d5640 5 monclient: authenticate success, global_id 20386 2023-07-29T14:05:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f106d5640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.696+0000 7f4f106d5640 4 set_mon_vals no callback set 2023-07-29T14:05:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.697+0000 7f4f106d5640 10 monclient: shutdown 2023-07-29T14:05:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.697+0000 7f4f106d5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.697+0000 7f4f106d5640 1 -- 172.21.15.40:0/3496382980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f08149520 msgr2=0x7f4f0814b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.697+0000 7f4f106d5640 1 --2- 172.21.15.40:0/3496382980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f08149520 0x7f4f0814b900 secure :-1 s=READY pgs=4786 cs=0 l=1 rev1=1 crypto rx=0x7f4f08070a10 tx=0x7f4f0002be40 comp rx=0 tx=0).stop 2023-07-29T14:05:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.697+0000 7f4f106d5640 1 -- 172.21.15.40:0/3496382980 shutdown_connections 2023-07-29T14:05:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.697+0000 7f4f106d5640 1 --2- 172.21.15.40:0/3496382980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f08149520 0x7f4f0814b900 unknown :-1 s=CLOSED pgs=4786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.697+0000 7f4f106d5640 1 --2- 172.21.15.40:0/3496382980 >> v2:172.21.15.40:3300/0 conn(0x7f4f08140c20 0x7f4f08141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.697+0000 7f4f106d5640 1 -- 172.21.15.40:0/3496382980 >> 172.21.15.40:0/3496382980 conn(0x7f4f08065920 msgr2=0x7f4f08065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.697+0000 7f4f106d5640 1 -- 172.21.15.40:0/3496382980 shutdown_connections 2023-07-29T14:05:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.697+0000 7f4f106d5640 1 -- 172.21.15.40:0/3496382980 wait complete. 2023-07-29T14:05:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.698+0000 7f4f106d5640 10 monclient: build_initial_monmap 2023-07-29T14:05:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.698+0000 7f4f106d5640 10 monclient: monmap: 2023-07-29T14:05:51.711 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:51.711 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:51.711 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:51.711 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:51.712 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:51.712 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.712 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:51.713 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:51.713 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:51.713 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.698+0000 7f4f106d5640 1 librados: starting msgr at 2023-07-29T14:05:51.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.698+0000 7f4f106d5640 1 librados: starting objecter 2023-07-29T14:05:51.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.698+0000 7f4f106d5640 1 Processor -- start 2023-07-29T14:05:51.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.698+0000 7f4f106d5640 1 -- start start 2023-07-29T14:05:51.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.698+0000 7f4f106d5640 1 librados: setting wanted keys 2023-07-29T14:05:51.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.698+0000 7f4f106d5640 1 librados: calling monclient init 2023-07-29T14:05:51.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.698+0000 7f4f106d5640 10 monclient: init 2023-07-29T14:05:51.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:51.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T14:05:51.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f080b3a60 0x7f4f08138850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 10 monclient(hunting): picked mon.noname-c con 0x7f4f080b3a60 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:51.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f08140c20 0x7f4f08144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 10 monclient(hunting): picked mon.noname-a con 0x7f4f08140c20 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:51.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f08149520 0x7f4f081d8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 10 monclient(hunting): picked mon.noname-b con 0x7f4f08149520 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:51.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08143f00 con 0x7f4f08140c20 2023-07-29T14:05:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08144170 con 0x7f4f08149520 2023-07-29T14:05:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08144c40 con 0x7f4f080b3a60 2023-07-29T14:05:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f106d5640 10 monclient(hunting): authenticate will time out at 3673.831387s 2023-07-29T14:05:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.699+0000 7f4f0e471640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f080b3a60 0x7f4f08138850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0e471640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f080b3a60 0x7f4f08138850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56656/0 (socket says 172.21.15.40:56656) 2023-07-29T14:05:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0e471640 1 -- 172.21.15.40:0/65380714 learned_addr learned my addr 172.21.15.40:0/65380714 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0e471640 10 monclient(hunting): get_auth_request con 0x7f4f080b3a60 auth_method 0 2023-07-29T14:05:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0e471640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0e471640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0e471640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0ec72640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f08149520 0x7f4f081d8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0e471640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0e471640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0e471640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0ec72640 10 monclient(hunting): get_auth_request con 0x7f4f08149520 auth_method 0 2023-07-29T14:05:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0ec72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0ec72640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.700+0000 7f4f0ec72640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.701+0000 7f4f0ec72640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.701+0000 7f4f0ec72640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.701+0000 7f4f0ec72640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.703+0000 7f4f0e471640 10 monclient(hunting): handle_auth_done global_id 16403 payload 824 2023-07-29T14:05:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.703+0000 7f4f0e471640 1 -- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f08149520 msgr2=0x7f4f081d8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.703+0000 7f4f0e471640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f08149520 0x7f4f081d8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.703+0000 7f4f0e471640 1 -- 172.21.15.40:0/65380714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f08140c20 msgr2=0x7f4f08144590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.703+0000 7f4f0e471640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f08140c20 0x7f4f08144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.703+0000 7f4f0e471640 10 monclient: _finish_hunting 0 2023-07-29T14:05:51.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.703+0000 7f4f0e471640 1 monclient: found mon.noname-c 2023-07-29T14:05:51.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.703+0000 7f4f0e471640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:51.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.703+0000 7f4f0e471640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.703+0000 7f4f0e471640 1 -- 172.21.15.40:0/65380714 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f00009a70 con 0x7f4f080b3a60 2023-07-29T14:05:51.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.705+0000 7f4f0e471640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f080b3a60 0x7f4f08138850 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c400 tx=0x7f4f0400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:51.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.705+0000 7f4efeffd640 10 client.?.objecter ms_handle_connect 0x7f4f080b3a60 2023-07-29T14:05:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.705+0000 7f4efeffd640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 1 -- 172.21.15.40:0/65380714 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04005ce0 con 0x7f4f080b3a60 2023-07-29T14:05:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 10 monclient: dump: 2023-07-29T14:05:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.732 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 10 monclient: _finish_auth 0 2023-07-29T14:05:51.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:51.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:51.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 1 -- 172.21.15.40:0/65380714 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f04005e40 con 0x7f4f080b3a60 2023-07-29T14:05:51.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:51.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 1 -- 172.21.15.40:0/65380714 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040072d0 con 0x7f4f080b3a60 2023-07-29T14:05:51.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:51.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efeffd640 10 monclient: dump: 2023-07-29T14:05:51.734 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.734 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.734 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.734 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.736 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.737 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4f106d5640 5 monclient: authenticate success, global_id 16403 2023-07-29T14:05:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4efffff640 4 set_mon_vals no callback set 2023-07-29T14:05:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4f106d5640 10 monclient: _renew_subs 2023-07-29T14:05:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4f106d5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4f106d5640 1 -- 172.21.15.40:0/65380714 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0813a900 con 0x7f4f080b3a60 2023-07-29T14:05:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4f106d5640 10 client.16403.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4f106d5640 10 monclient: _renew_subs 2023-07-29T14:05:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4f106d5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4f106d5640 1 -- 172.21.15.40:0/65380714 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f081d91b0 con 0x7f4f080b3a60 2023-07-29T14:05:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.709+0000 7f4f106d5640 1 librados: init done 2023-07-29T14:05:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.710+0000 7f4f106d5640 10 client.16403.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.710+0000 7f4f106d5640 15 client.16403.objecter _session_command_op_assign -1 1 2023-07-29T14:05:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.710+0000 7f4f106d5640 20 client.16403.objecter _get_session osd=-1 returning homeless 2023-07-29T14:05:51.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.710+0000 7f4f106d5640 10 client.16403.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:51.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.710+0000 7f4f106d5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.710+0000 7f4f106d5640 1 -- 172.21.15.40:0/65380714 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4f080a98e0 con 0x7f4f080b3a60 2023-07-29T14:05:51.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.729+0000 7f4efeffd640 1 -- 172.21.15.40:0/65380714 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f4f04007430 con 0x7f4f080b3a60 2023-07-29T14:05:51.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.730+0000 7f4efeffd640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f4eec06bfe0 0x7f4eec06e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.730+0000 7f4f0dc70640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f4eec06bfe0 0x7f4eec06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.730+0000 7f4f0dc70640 10 monclient: get_auth_request con 0x7f4eec06bfe0 auth_method 0 2023-07-29T14:05:51.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.731+0000 7f4efeffd640 10 client.16403.objecter ms_handle_connect 0x7f4eec06bfe0 2023-07-29T14:05:51.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.731+0000 7f4f0dc70640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f4eec06bfe0 0x7f4eec06e490 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8006a90 tx=0x7f4ef8006a20 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:51.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 1 -- 172.21.15.40:0/65380714 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f4f0405b7f0 con 0x7f4f080b3a60 2023-07-29T14:05:51.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 10 client.16403.objecter ms_dispatch 0x7f4f08000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:51.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 3 client.16403.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:51.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 3 client.16403.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:51.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 10 client.16403.objecter checking command 1 2023-07-29T14:05:51.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] conn(0x7f4eec073820 0x7f4eec075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 20 client.16403.objecter _get_session s=0x7f4eec073150 osd=4 3 2023-07-29T14:05:51.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 20 client.16403.objecter put_session s=0x7f4eec073150 osd=4 3 2023-07-29T14:05:51.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 15 client.16403.objecter _session_command_op_remove -1 1 2023-07-29T14:05:51.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 20 client.16403.objecter _get_session s=0x7f4eec073150 osd=4 3 2023-07-29T14:05:51.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 20 client.16403.objecter get_session s=0x7f4eec073150 osd=4 3 2023-07-29T14:05:51.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 15 client.16403.objecter _session_command_op_assign 4 1 2023-07-29T14:05:51.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 20 client.16403.objecter put_session s=0x7f4eec073150 osd=4 4 2023-07-29T14:05:51.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 10 client.16403.objecter _send_command 1 2023-07-29T14:05:51.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 1 -- 172.21.15.40:0/65380714 --> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4eec076380 con 0x7f4eec073820 2023-07-29T14:05:51.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 20 client.16403.objecter dump_active .. 0 homeless 2023-07-29T14:05:51.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 1 -- 172.21.15.40:0/65380714 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=1004) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f04053c20 con 0x7f4f080b3a60 2023-07-29T14:05:51.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.735+0000 7f4efeffd640 10 monclient: handle_get_version_reply finishing 1 version 1004 2023-07-29T14:05:51.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.736+0000 7f4f0ec72640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] conn(0x7f4eec073820 0x7f4eec075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.736+0000 7f4f0ec72640 10 monclient: get_auth_request con 0x7f4eec073820 auth_method 0 2023-07-29T14:05:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.739+0000 7f4f0ec72640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] conn(0x7f4eec073820 0x7f4eec075cb0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.739+0000 7f4efeffd640 10 client.16403.objecter ms_handle_connect 0x7f4eec073820 2023-07-29T14:05:51.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.764+0000 7f4efeffd640 1 -- 172.21.15.40:0/65380714 <== osd.4 v2:172.21.15.45:6826/85183 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4eec076380 con 0x7f4eec073820 2023-07-29T14:05:51.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.764+0000 7f4efeffd640 10 client.16403.objecter ms_dispatch 0x7f4f08000b90 command_reply(tid 1: 0 ) v1 2023-07-29T14:05:51.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.764+0000 7f4efeffd640 10 client.16403.objecter _finish_command 1 = system:0 2023-07-29T14:05:51.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.764+0000 7f4efeffd640 20 client.16403.objecter put_session s=0x7f4eec073150 osd=4 4 2023-07-29T14:05:51.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.764+0000 7f4efeffd640 15 client.16403.objecter _session_command_op_remove 4 1 2023-07-29T14:05:51.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.778+0000 7f4f106d5640 10 client.16403.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-07-29T14:05:51.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.778+0000 7f4f106d5640 15 client.16403.objecter _session_command_op_assign -1 2 2023-07-29T14:05:51.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.778+0000 7f4f106d5640 20 client.16403.objecter _get_session s=0x7f4eec073150 osd=4 3 2023-07-29T14:05:51.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.778+0000 7f4f106d5640 20 client.16403.objecter put_session s=0x7f4eec073150 osd=4 3 2023-07-29T14:05:51.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.778+0000 7f4f106d5640 20 client.16403.objecter _get_session s=0x7f4eec073150 osd=4 3 2023-07-29T14:05:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.778+0000 7f4f106d5640 15 client.16403.objecter _session_command_op_remove -1 2 2023-07-29T14:05:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.778+0000 7f4f106d5640 20 client.16403.objecter get_session s=0x7f4eec073150 osd=4 3 2023-07-29T14:05:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.778+0000 7f4f106d5640 15 client.16403.objecter _session_command_op_assign 4 2 2023-07-29T14:05:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.778+0000 7f4f106d5640 20 client.16403.objecter put_session s=0x7f4eec073150 osd=4 4 2023-07-29T14:05:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.778+0000 7f4f106d5640 10 client.16403.objecter _send_command 2 2023-07-29T14:05:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.778+0000 7f4f106d5640 1 -- 172.21.15.40:0/65380714 --> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4f080a9600 con 0x7f4eec073820 2023-07-29T14:05:51.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4efeffd640 1 -- 172.21.15.40:0/65380714 <== osd.4 v2:172.21.15.45:6826/85183 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4f0002cb10 con 0x7f4eec073820 2023-07-29T14:05:51.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4efeffd640 10 client.16403.objecter ms_dispatch 0x7f4f08000b90 command_reply(tid 2: 0 ) v1 2023-07-29T14:05:51.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4efeffd640 10 client.16403.objecter _finish_command 2 = system:0 2023-07-29T14:05:51.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4efeffd640 20 client.16403.objecter put_session s=0x7f4eec073150 osd=4 4 2023-07-29T14:05:51.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4efeffd640 15 client.16403.objecter _session_command_op_remove 4 2 2023-07-29T14:05:51.794 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-29T14:05:51.794 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-29T14:05:51.794 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-29T14:05:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 10 librados: watch_flush enter 2023-07-29T14:05:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 10 librados: watch_flush exit 2023-07-29T14:05:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 10 client.16403.objecter close_session for osd.4 2023-07-29T14:05:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 1 -- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] conn(0x7f4eec073820 msgr2=0x7f4eec075cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:6826/85183,v1:172.21.15.45:6827/85183] conn(0x7f4eec073820 0x7f4eec075cb0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 20 client.16403.objecter put_session s=0x7f4eec073150 osd=4 1 2023-07-29T14:05:51.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 20 client.16403.objecter shutdown clearing up homeless session... 2023-07-29T14:05:51.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 10 client.16403.objecter successfully canceled tick 2023-07-29T14:05:51.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 1 -- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f4eec06bfe0 msgr2=0x7f4eec06e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f4eec06bfe0 0x7f4eec06e490 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8006a90 tx=0x7f4ef8006a20 comp rx=0 tx=0).stop 2023-07-29T14:05:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 10 monclient: shutdown 2023-07-29T14:05:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 1 -- 172.21.15.40:0/65380714 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f080b3a60 msgr2=0x7f4f08138850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 1 --2- 172.21.15.40:0/65380714 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4f080b3a60 0x7f4f08138850 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c400 tx=0x7f4f0400c8c0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f0ec72640 1 -- 172.21.15.40:0/65380714 reap_dead start 2023-07-29T14:05:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 1 -- 172.21.15.40:0/65380714 shutdown_connections 2023-07-29T14:05:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 1 -- 172.21.15.40:0/65380714 >> 172.21.15.40:0/65380714 conn(0x7f4f08065920 msgr2=0x7f4f0813bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 1 -- 172.21.15.40:0/65380714 shutdown_connections 2023-07-29T14:05:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 1 -- 172.21.15.40:0/65380714 wait complete. 2023-07-29T14:05:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.793+0000 7f4f106d5640 1 librados: shutdown 2023-07-29T14:05:51.804 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-07-29T14:05:51.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.916+0000 7fbf39908640 10 monclient: get_monmap_and_config 2023-07-29T14:05:51.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.916+0000 7fbf39908640 10 monclient: build_initial_monmap 2023-07-29T14:05:51.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.916+0000 7fbf39908640 10 monclient: monmap: 2023-07-29T14:05:51.919 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:51.919 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.919 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:51.916644+0000 2023-07-29T14:05:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:51.916644+0000 2023-07-29T14:05:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:51.921 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 1 Processor -- start 2023-07-29T14:05:51.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 1 -- start start 2023-07-29T14:05:51.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 10 monclient: init 2023-07-29T14:05:51.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:51.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2023-07-29T14:05:51.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 0x7fbf340aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 10 monclient(hunting): picked mon.noname-b con 0x7fbf340b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:51.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbf3413dde0 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:51.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbf341445e0 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:51.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf340b7870 con 0x7fbf3413dde0 2023-07-29T14:05:51.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf340a91b0 con 0x7fbf341445e0 2023-07-29T14:05:51.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf340a9490 con 0x7fbf340b74a0 2023-07-29T14:05:51.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:51.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.917+0000 7fbf39908640 10 monclient(hunting): authenticate will time out at 3674.049767s 2023-07-29T14:05:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.918+0000 7fbf32ffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 0x7fbf340aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.918+0000 7fbf327fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbf3413dde0 0x7fbf3413e1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38654/0 (socket says 172.21.15.40:38654) 2023-07-29T14:05:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.918+0000 7fbf327fc640 1 -- 172.21.15.40:0/3970478658 learned_addr learned my addr 172.21.15.40:0/3970478658 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.918+0000 7fbf32ffd640 10 monclient(hunting): get_auth_request con 0x7fbf340b74a0 auth_method 0 2023-07-29T14:05:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.918+0000 7fbf32ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.918+0000 7fbf32ffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.918+0000 7fbf32ffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.919+0000 7fbf32ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.919+0000 7fbf32ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.919+0000 7fbf32ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.920+0000 7fbf32ffd640 10 monclient(hunting): handle_auth_done global_id 16406 payload 274 2023-07-29T14:05:51.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.920+0000 7fbf32ffd640 1 -- 172.21.15.40:0/3970478658 >> v1:172.21.15.45:6789/0 conn(0x7fbf341445e0 legacy=0x7fbf34144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.920+0000 7fbf32ffd640 1 -- 172.21.15.40:0/3970478658 >> v1:172.21.15.40:6789/0 conn(0x7fbf3413dde0 legacy=0x7fbf3413e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.920+0000 7fbf32ffd640 10 monclient: _finish_hunting 0 2023-07-29T14:05:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.920+0000 7fbf32ffd640 1 monclient: found mon.noname-b 2023-07-29T14:05:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.920+0000 7fbf32ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.920+0000 7fbf32ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.920+0000 7fbf32ffd640 1 -- 172.21.15.40:0/3970478658 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340a9770 con 0x7fbf340b74a0 2023-07-29T14:05:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.923+0000 7fbf32ffd640 1 --2- 172.21.15.40:0/3970478658 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 0x7fbf340aa150 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009e10 tx=0x7fbf2402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a67089bbb31625d server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 1 -- 172.21.15.40:0/3970478658 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2402cc80 con 0x7fbf340b74a0 2023-07-29T14:05:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 10 monclient: dump: 2023-07-29T14:05:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.937 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 10 monclient: _finish_auth 0 2023-07-29T14:05:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 10 monclient: _check_auth_tickets 2023-07-29T14:05:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 1 -- 172.21.15.40:0/3970478658 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2402cde0 con 0x7fbf340b74a0 2023-07-29T14:05:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 1 -- 172.21.15.40:0/3970478658 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24034660 con 0x7fbf340b74a0 2023-07-29T14:05:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf39908640 5 monclient: authenticate success, global_id 16406 2023-07-29T14:05:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf39908640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf39908640 4 set_mon_vals no callback set 2023-07-29T14:05:51.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:51.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.926+0000 7fbf31ffb640 10 monclient: dump: 2023-07-29T14:05:51.939 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.939 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.939 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.939 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.940 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.940 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.941 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.941 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.941 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.942 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.927+0000 7fbf39908640 10 monclient: shutdown 2023-07-29T14:05:51.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.927+0000 7fbf39908640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:51.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.927+0000 7fbf39908640 1 -- 172.21.15.40:0/3970478658 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 msgr2=0x7fbf340aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.927+0000 7fbf39908640 1 --2- 172.21.15.40:0/3970478658 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 0x7fbf340aa150 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009e10 tx=0x7fbf2402bed0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.927+0000 7fbf39908640 1 -- 172.21.15.40:0/3970478658 shutdown_connections 2023-07-29T14:05:51.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.927+0000 7fbf39908640 1 --2- 172.21.15.40:0/3970478658 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 0x7fbf340aa150 unknown :-1 s=CLOSED pgs=3233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.927+0000 7fbf39908640 1 -- 172.21.15.40:0/3970478658 >> 172.21.15.40:0/3970478658 conn(0x7fbf34065b00 msgr2=0x7fbf34065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:51.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.927+0000 7fbf39908640 1 -- 172.21.15.40:0/3970478658 shutdown_connections 2023-07-29T14:05:51.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.927+0000 7fbf39908640 1 -- 172.21.15.40:0/3970478658 wait complete. 2023-07-29T14:05:51.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.928+0000 7fbf39908640 10 monclient: build_initial_monmap 2023-07-29T14:05:51.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.928+0000 7fbf39908640 10 monclient: monmap: 2023-07-29T14:05:51.943 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:51.946 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.928+0000 7fbf39908640 1 librados: starting msgr at 2023-07-29T14:05:51.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.928+0000 7fbf39908640 1 librados: starting objecter 2023-07-29T14:05:51.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.928+0000 7fbf39908640 1 Processor -- start 2023-07-29T14:05:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.928+0000 7fbf39908640 1 -- start start 2023-07-29T14:05:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.928+0000 7fbf39908640 1 librados: setting wanted keys 2023-07-29T14:05:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.928+0000 7fbf39908640 1 librados: calling monclient init 2023-07-29T14:05:51.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.928+0000 7fbf39908640 10 monclient: init 2023-07-29T14:05:51.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:51.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T14:05:51.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 0x7fbf3413ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 10 monclient(hunting): picked mon.noname-c con 0x7fbf340b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:51.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3413dde0 0x7fbf340b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 10 monclient(hunting): picked mon.noname-a con 0x7fbf3413dde0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:51.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf341445e0 0x7fbf340b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 10 monclient(hunting): picked mon.noname-b con 0x7fbf341445e0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:51.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf340b0270 con 0x7fbf3413dde0 2023-07-29T14:05:51.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf34139d90 con 0x7fbf341445e0 2023-07-29T14:05:51.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:51.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf34139f50 con 0x7fbf340b74a0 2023-07-29T14:05:51.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:51.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf39908640 10 monclient(hunting): authenticate will time out at 3674.061626s 2023-07-29T14:05:51.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.929+0000 7fbf32ffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 0x7fbf3413ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf341445e0 0x7fbf340b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3413dde0 0x7fbf340b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf32ffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 0x7fbf3413ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56678/0 (socket says 172.21.15.40:56678) 2023-07-29T14:05:51.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf32ffd640 1 -- 172.21.15.40:0/2963048924 learned_addr learned my addr 172.21.15.40:0/2963048924 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:51.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf32ffd640 10 monclient(hunting): get_auth_request con 0x7fbf340b74a0 auth_method 0 2023-07-29T14:05:51.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf32ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf32ffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf32ffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf337fe640 10 monclient(hunting): get_auth_request con 0x7fbf341445e0 auth_method 0 2023-07-29T14:05:51.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf337fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf337fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf327fc640 10 monclient(hunting): get_auth_request con 0x7fbf3413dde0 auth_method 0 2023-07-29T14:05:51.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf327fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:51.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf327fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:51.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf327fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:51.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf32ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf32ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf32ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.930+0000 7fbf337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.931+0000 7fbf327fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.931+0000 7fbf327fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.931+0000 7fbf327fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.933+0000 7fbf32ffd640 10 monclient(hunting): handle_auth_done global_id 16409 payload 824 2023-07-29T14:05:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.933+0000 7fbf32ffd640 1 -- 172.21.15.40:0/2963048924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf341445e0 msgr2=0x7fbf340b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.933+0000 7fbf32ffd640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf341445e0 0x7fbf340b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.933+0000 7fbf32ffd640 1 -- 172.21.15.40:0/2963048924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3413dde0 msgr2=0x7fbf340b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.933+0000 7fbf32ffd640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3413dde0 0x7fbf340b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.933+0000 7fbf32ffd640 10 monclient: _finish_hunting 0 2023-07-29T14:05:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.933+0000 7fbf32ffd640 1 monclient: found mon.noname-c 2023-07-29T14:05:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.933+0000 7fbf32ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.933+0000 7fbf32ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.933+0000 7fbf32ffd640 1 -- 172.21.15.40:0/2963048924 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf24009a70 con 0x7fbf340b74a0 2023-07-29T14:05:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.936+0000 7fbf32ffd640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 0x7fbf3413ac40 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 crypto rx=0x7fbf24000f80 tx=0x7fbf24004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.936+0000 7fbf1bfff640 10 client.?.objecter ms_handle_connect 0x7fbf340b74a0 2023-07-29T14:05:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.936+0000 7fbf1bfff640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 1 -- 172.21.15.40:0/2963048924 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2403e610 con 0x7fbf340b74a0 2023-07-29T14:05:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 10 monclient: dump: 2023-07-29T14:05:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.962 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 10 monclient: _finish_auth 0 2023-07-29T14:05:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 10 monclient: _check_auth_tickets 2023-07-29T14:05:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 1 -- 172.21.15.40:0/2963048924 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2403e770 con 0x7fbf340b74a0 2023-07-29T14:05:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 1 -- 172.21.15.40:0/2963048924 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2403eb20 con 0x7fbf340b74a0 2023-07-29T14:05:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf1bfff640 10 monclient: dump: 2023-07-29T14:05:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:51.964 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf39908640 5 monclient: authenticate success, global_id 16409 2023-07-29T14:05:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf317fa640 4 set_mon_vals no callback set 2023-07-29T14:05:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf39908640 10 monclient: _renew_subs 2023-07-29T14:05:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf39908640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf39908640 1 -- 172.21.15.40:0/2963048924 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3413b180 con 0x7fbf340b74a0 2023-07-29T14:05:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf39908640 10 client.16409.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf39908640 10 monclient: _renew_subs 2023-07-29T14:05:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf39908640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf39908640 1 -- 172.21.15.40:0/2963048924 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf341d5a10 con 0x7fbf340b74a0 2023-07-29T14:05:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.940+0000 7fbf39908640 1 librados: init done 2023-07-29T14:05:51.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.941+0000 7fbf39908640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:51.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.941+0000 7fbf39908640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:51.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.941+0000 7fbf39908640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:51.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.941+0000 7fbf39908640 1 -- 172.21.15.40:0/2963048924 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3413b340 con 0x7fbf340b74a0 2023-07-29T14:05:51.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.962+0000 7fbf1bfff640 1 -- 172.21.15.40:0/2963048924 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fbf2403b050 con 0x7fbf340b74a0 2023-07-29T14:05:51.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.963+0000 7fbf1bfff640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbf2006bfb0 0x7fbf2006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:51.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.963+0000 7fbf327fc640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbf2006bfb0 0x7fbf2006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:51.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.963+0000 7fbf327fc640 10 monclient: get_auth_request con 0x7fbf2006bfb0 auth_method 0 2023-07-29T14:05:51.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.963+0000 7fbf327fc640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbf2006bfb0 0x7fbf2006e460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbf340b1310 tx=0x7fbf28009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:51.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:51.963+0000 7fbf1bfff640 10 client.16409.objecter ms_handle_connect 0x7fbf2006bfb0 2023-07-29T14:05:52.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.227+0000 7fbf1bfff640 1 -- 172.21.15.40:0/2963048924 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7fbf2407e710 con 0x7fbf340b74a0 2023-07-29T14:05:52.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.227+0000 7fbf1bfff640 10 client.16409.objecter ms_dispatch 0x7fbf34000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:52.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.227+0000 7fbf1bfff640 3 client.16409.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:52.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.227+0000 7fbf1bfff640 3 client.16409.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:52.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.227+0000 7fbf1bfff640 20 client.16409.objecter dump_active .. 0 homeless 2023-07-29T14:05:52.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.227+0000 7fbf1bfff640 1 -- 172.21.15.40:0/2963048924 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7fbf24076b40 con 0x7fbf340b74a0 2023-07-29T14:05:52.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.227+0000 7fbf1bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:52.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.227+0000 7fbf1bfff640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.408+0000 7fbf39908640 1 -- 172.21.15.40:0/2963048924 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf34145170 con 0x7fbf2006bfb0 2023-07-29T14:05:52.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.409+0000 7fbf1bfff640 1 -- 172.21.15.40:0/2963048924 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbf2800f820 con 0x7fbf2006bfb0 2023-07-29T14:05:52.409 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2023-07-29T14:05:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 10 librados: watch_flush enter 2023-07-29T14:05:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 10 librados: watch_flush exit 2023-07-29T14:05:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 20 client.16409.objecter shutdown clearing up homeless session... 2023-07-29T14:05:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 10 client.16409.objecter successfully canceled tick 2023-07-29T14:05:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 -- 172.21.15.40:0/2963048924 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbf2006bfb0 msgr2=0x7fbf2006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbf2006bfb0 0x7fbf2006e460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbf340b1310 tx=0x7fbf28009040 comp rx=0 tx=0).stop 2023-07-29T14:05:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 10 monclient: shutdown 2023-07-29T14:05:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 -- 172.21.15.40:0/2963048924 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 msgr2=0x7fbf3413ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 0x7fbf3413ac40 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 crypto rx=0x7fbf24000f80 tx=0x7fbf24004a40 comp rx=0 tx=0).stop 2023-07-29T14:05:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 -- 172.21.15.40:0/2963048924 shutdown_connections 2023-07-29T14:05:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fbf2006bfb0 0x7fbf2006e460 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf341445e0 0x7fbf340b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3413dde0 0x7fbf340b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 --2- 172.21.15.40:0/2963048924 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fbf340b74a0 0x7fbf3413ac40 unknown :-1 s=CLOSED pgs=3234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 -- 172.21.15.40:0/2963048924 >> 172.21.15.40:0/2963048924 conn(0x7fbf34065b00 msgr2=0x7fbf3406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 -- 172.21.15.40:0/2963048924 shutdown_connections 2023-07-29T14:05:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 -- 172.21.15.40:0/2963048924 wait complete. 2023-07-29T14:05:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.411+0000 7fbf39908640 1 librados: shutdown 2023-07-29T14:05:52.423 INFO:tasks.ceph:Scrubbing osd.5 2023-07-29T14:05:52.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-07-29T14:05:52.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.538+0000 7f97c54c7640 10 monclient: get_monmap_and_config 2023-07-29T14:05:52.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.538+0000 7f97c54c7640 10 monclient: build_initial_monmap 2023-07-29T14:05:52.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.538+0000 7f97c54c7640 10 monclient: monmap: 2023-07-29T14:05:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:52.538847+0000 2023-07-29T14:05:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:52.538847+0000 2023-07-29T14:05:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:52.540 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.539+0000 7f97c54c7640 1 Processor -- start 2023-07-29T14:05:52.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.539+0000 7f97c54c7640 1 -- start start 2023-07-29T14:05:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.539+0000 7f97c54c7640 10 monclient: init 2023-07-29T14:05:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.539+0000 7f97c54c7640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.539+0000 7f97c54c7640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2023-07-29T14:05:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f97c00b3550 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97c00b3920 0x7f97c00af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 10 monclient(hunting): picked mon.noname-a con 0x7f97c00b3920 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b0170 0x7f97c00b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 10 monclient(hunting): picked mon.noname-b con 0x7f97c00b0170 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:52.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c0141e50 con 0x7f97c00b3550 2023-07-29T14:05:52.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:52.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c013ec30 con 0x7f97c00b3920 2023-07-29T14:05:52.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:52.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00afd90 con 0x7f97c00b0170 2023-07-29T14:05:52.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:52.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97c54c7640 10 monclient(hunting): authenticate will time out at 3674.672151s 2023-07-29T14:05:52.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b0170 0x7f97c00b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:52.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97b7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97c00b3920 0x7f97c00af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:52.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b0170 0x7f97c00b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56680/0 (socket says 172.21.15.40:56680) 2023-07-29T14:05:52.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97bf7fe640 1 -- 172.21.15.40:0/3267999093 learned_addr learned my addr 172.21.15.40:0/3267999093 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:52.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97bf7fe640 10 monclient(hunting): get_auth_request con 0x7f97c00b0170 auth_method 0 2023-07-29T14:05:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97bf7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.540+0000 7f97bf7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.541+0000 7f97b7fff640 10 monclient(hunting): get_auth_request con 0x7f97c00b3920 auth_method 0 2023-07-29T14:05:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.541+0000 7f97b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.541+0000 7f97b7fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.541+0000 7f97b7fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.541+0000 7f97bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.541+0000 7f97bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.541+0000 7f97bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.541+0000 7f97b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.541+0000 7f97b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.541+0000 7f97b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.543+0000 7f97bf7fe640 10 monclient(hunting): handle_auth_done global_id 16412 payload 274 2023-07-29T14:05:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.543+0000 7f97bf7fe640 1 -- 172.21.15.40:0/3267999093 >> v2:172.21.15.40:3300/0 conn(0x7f97c00b3920 msgr2=0x7f97c00af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:52.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.543+0000 7f97bf7fe640 1 --2- 172.21.15.40:0/3267999093 >> v2:172.21.15.40:3300/0 conn(0x7f97c00b3920 0x7f97c00af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.543+0000 7f97bf7fe640 1 -- 172.21.15.40:0/3267999093 >> v1:172.21.15.45:6789/0 conn(0x7f97c00b3550 legacy=0x7f97c0141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.543+0000 7f97bf7fe640 10 monclient: _finish_hunting 0 2023-07-29T14:05:52.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.543+0000 7f97bf7fe640 1 monclient: found mon.noname-b 2023-07-29T14:05:52.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.543+0000 7f97bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:52.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.543+0000 7f97bf7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:52.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.543+0000 7f97bf7fe640 1 -- 172.21.15.40:0/3267999093 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c00b0b10 con 0x7f97c00b0170 2023-07-29T14:05:52.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.545+0000 7f97bf7fe640 1 --2- 172.21.15.40:0/3267999093 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b0170 0x7f97c00b05d0 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009e50 tx=0x7f97b802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e8c04441aa8bab3 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.548+0000 7f97be7fc640 1 -- 172.21.15.40:0/3267999093 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b802ca70 con 0x7f97c00b0170 2023-07-29T14:05:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.548+0000 7f97be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97be7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97be7fc640 10 monclient: dump: 2023-07-29T14:05:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:52.557 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97be7fc640 10 monclient: _finish_auth 0 2023-07-29T14:05:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97be7fc640 10 monclient: _check_auth_tickets 2023-07-29T14:05:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 5 monclient: authenticate success, global_id 16412 2023-07-29T14:05:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97be7fc640 1 -- 172.21.15.40:0/3267999093 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b802cbd0 con 0x7f97c00b0170 2023-07-29T14:05:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97be7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 4 set_mon_vals no callback set 2023-07-29T14:05:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97be7fc640 1 -- 172.21.15.40:0/3267999093 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b80343f0 con 0x7f97c00b0170 2023-07-29T14:05:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97be7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97be7fc640 10 monclient: dump: 2023-07-29T14:05:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:52.560 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 10 monclient: shutdown 2023-07-29T14:05:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 1 -- 172.21.15.40:0/3267999093 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b0170 msgr2=0x7f97c00b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 1 --2- 172.21.15.40:0/3267999093 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b0170 0x7f97c00b05d0 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009e50 tx=0x7f97b802bed0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 1 -- 172.21.15.40:0/3267999093 shutdown_connections 2023-07-29T14:05:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 1 --2- 172.21.15.40:0/3267999093 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b0170 0x7f97c00b05d0 unknown :-1 s=CLOSED pgs=3235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 1 --2- 172.21.15.40:0/3267999093 >> v2:172.21.15.40:3300/0 conn(0x7f97c00b3920 0x7f97c00af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 1 -- 172.21.15.40:0/3267999093 >> 172.21.15.40:0/3267999093 conn(0x7f97c0065920 msgr2=0x7f97c0065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 1 -- 172.21.15.40:0/3267999093 shutdown_connections 2023-07-29T14:05:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.549+0000 7f97c54c7640 1 -- 172.21.15.40:0/3267999093 wait complete. 2023-07-29T14:05:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.550+0000 7f97c54c7640 10 monclient: build_initial_monmap 2023-07-29T14:05:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.550+0000 7f97c54c7640 10 monclient: monmap: 2023-07-29T14:05:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:52.563 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.563 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:52.564 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:52.564 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:52.565 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.550+0000 7f97c54c7640 1 librados: starting msgr at 2023-07-29T14:05:52.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.550+0000 7f97c54c7640 1 librados: starting objecter 2023-07-29T14:05:52.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.550+0000 7f97c54c7640 1 Processor -- start 2023-07-29T14:05:52.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.550+0000 7f97c54c7640 1 -- start start 2023-07-29T14:05:52.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.550+0000 7f97c54c7640 1 librados: setting wanted keys 2023-07-29T14:05:52.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.550+0000 7f97c54c7640 1 librados: calling monclient init 2023-07-29T14:05:52.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.550+0000 7f97c54c7640 10 monclient: init 2023-07-29T14:05:52.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:52.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T14:05:52.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97c00b0170 0x7f97c01d6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 10 monclient(hunting): picked mon.noname-b con 0x7f97c00b0170 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:52.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b3550 0x7f97c01d69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 10 monclient(hunting): picked mon.noname-c con 0x7f97c00b3550 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:52.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97c00b3920 0x7f97c00ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 10 monclient(hunting): picked mon.noname-a con 0x7f97c00b3920 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:52.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:52.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c01d5fd0 con 0x7f97c00b3920 2023-07-29T14:05:52.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:52.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00ab7f0 con 0x7f97c00b0170 2023-07-29T14:05:52.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:52.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00aae30 con 0x7f97c00b3550 2023-07-29T14:05:52.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:52.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97c54c7640 10 monclient(hunting): authenticate will time out at 3674.683510s 2023-07-29T14:05:52.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.551+0000 7f97b7fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b3550 0x7f97c01d69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:52.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97c00b3920 0x7f97c00ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97b7fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b3550 0x7f97c01d69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56686/0 (socket says 172.21.15.40:56686) 2023-07-29T14:05:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97b7fff640 1 -- 172.21.15.40:0/2861576484 learned_addr learned my addr 172.21.15.40:0/2861576484 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97b7fff640 10 monclient(hunting): get_auth_request con 0x7f97c00b3550 auth_method 0 2023-07-29T14:05:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97b7fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97b7fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97beffd640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97c00b0170 0x7f97c01d6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:52.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97bf7fe640 10 monclient(hunting): get_auth_request con 0x7f97c00b3920 auth_method 0 2023-07-29T14:05:52.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:52.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97bf7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:52.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97bf7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:52.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97beffd640 10 monclient(hunting): get_auth_request con 0x7f97c00b0170 auth_method 0 2023-07-29T14:05:52.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:52.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97beffd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:52.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97beffd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:52.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:52.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:52.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.552+0000 7f97b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:52.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.553+0000 7f97bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:52.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.553+0000 7f97bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:52.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.553+0000 7f97bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:52.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.553+0000 7f97beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:52.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.553+0000 7f97beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:52.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.553+0000 7f97beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.555+0000 7f97b7fff640 10 monclient(hunting): handle_auth_done global_id 16415 payload 824 2023-07-29T14:05:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.555+0000 7f97b7fff640 1 -- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97c00b0170 msgr2=0x7f97c01d6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.555+0000 7f97b7fff640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97c00b0170 0x7f97c01d6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.555+0000 7f97b7fff640 1 -- 172.21.15.40:0/2861576484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97c00b3920 msgr2=0x7f97c00ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.555+0000 7f97b7fff640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97c00b3920 0x7f97c00ac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.555+0000 7f97b7fff640 10 monclient: _finish_hunting 0 2023-07-29T14:05:52.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.555+0000 7f97b7fff640 1 monclient: found mon.noname-c 2023-07-29T14:05:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.555+0000 7f97b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.555+0000 7f97b7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:52.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.555+0000 7f97b7fff640 1 -- 172.21.15.40:0/2861576484 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8009b30 con 0x7f97c00b3550 2023-07-29T14:05:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.558+0000 7f97b7fff640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b3550 0x7f97c01d69a0 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 crypto rx=0x7f97a800e8a0 tx=0x7f97a800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.558+0000 7f97b77fe640 10 client.?.objecter ms_handle_connect 0x7f97c00b3550 2023-07-29T14:05:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.558+0000 7f97b77fe640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 1 -- 172.21.15.40:0/2861576484 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a80094f0 con 0x7f97c00b3550 2023-07-29T14:05:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 10 monclient: dump: 2023-07-29T14:05:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:52.588 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 10 monclient: _finish_auth 0 2023-07-29T14:05:52.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 10 monclient: _check_auth_tickets 2023-07-29T14:05:52.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:52.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 1 -- 172.21.15.40:0/2861576484 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a800c940 con 0x7f97c00b3550 2023-07-29T14:05:52.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:52.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 1 -- 172.21.15.40:0/2861576484 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a800ccf0 con 0x7f97c00b3550 2023-07-29T14:05:52.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:52.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:52.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97b77fe640 10 monclient: dump: 2023-07-29T14:05:52.591 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:52.591 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:52.592 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.592 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.592 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:52.593 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.593 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:52.593 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:52.594 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:52.594 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97c54c7640 5 monclient: authenticate success, global_id 16415 2023-07-29T14:05:52.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97c54c7640 10 monclient: _renew_subs 2023-07-29T14:05:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97bdffb640 4 set_mon_vals no callback set 2023-07-29T14:05:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97c54c7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.561+0000 7f97c54c7640 1 -- 172.21.15.40:0/2861576484 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c00a94a0 con 0x7f97c00b3550 2023-07-29T14:05:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.562+0000 7f97c54c7640 10 client.16415.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.562+0000 7f97c54c7640 10 monclient: _renew_subs 2023-07-29T14:05:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.562+0000 7f97c54c7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.562+0000 7f97c54c7640 1 -- 172.21.15.40:0/2861576484 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c00ac960 con 0x7f97c00b3550 2023-07-29T14:05:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.562+0000 7f97c54c7640 1 librados: init done 2023-07-29T14:05:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.562+0000 7f97c54c7640 10 client.16415.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.562+0000 7f97c54c7640 15 client.16415.objecter _session_command_op_assign -1 1 2023-07-29T14:05:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.562+0000 7f97c54c7640 20 client.16415.objecter _get_session osd=-1 returning homeless 2023-07-29T14:05:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.562+0000 7f97c54c7640 10 client.16415.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.562+0000 7f97c54c7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.562+0000 7f97c54c7640 1 -- 172.21.15.40:0/2861576484 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97c0140f40 con 0x7f97c00b3550 2023-07-29T14:05:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.583+0000 7f97b77fe640 1 -- 172.21.15.40:0/2861576484 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f97a8013470 con 0x7f97c00b3550 2023-07-29T14:05:52.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.583+0000 7f97b77fe640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f97a006bfb0 0x7f97a006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.583+0000 7f97beffd640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f97a006bfb0 0x7f97a006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:52.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.583+0000 7f97beffd640 10 monclient: get_auth_request con 0x7f97a006bfb0 auth_method 0 2023-07-29T14:05:52.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.584+0000 7f97b77fe640 10 client.16415.objecter ms_handle_connect 0x7f97a006bfb0 2023-07-29T14:05:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.584+0000 7f97beffd640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f97a006bfb0 0x7f97a006e460 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f97b0001780 tx=0x7f97b0009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 1 -- 172.21.15.40:0/2861576484 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f97a805bf30 con 0x7f97c00b3550 2023-07-29T14:05:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 10 client.16415.objecter ms_dispatch 0x7f97c0000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 3 client.16415.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 3 client.16415.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 10 client.16415.objecter checking command 1 2023-07-29T14:05:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] conn(0x7f97a00737f0 0x7f97a0075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 20 client.16415.objecter _get_session s=0x7f97a0073120 osd=5 3 2023-07-29T14:05:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 20 client.16415.objecter put_session s=0x7f97a0073120 osd=5 3 2023-07-29T14:05:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 15 client.16415.objecter _session_command_op_remove -1 1 2023-07-29T14:05:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 20 client.16415.objecter _get_session s=0x7f97a0073120 osd=5 3 2023-07-29T14:05:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 20 client.16415.objecter get_session s=0x7f97a0073120 osd=5 3 2023-07-29T14:05:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 15 client.16415.objecter _session_command_op_assign 5 1 2023-07-29T14:05:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 20 client.16415.objecter put_session s=0x7f97a0073120 osd=5 4 2023-07-29T14:05:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 10 client.16415.objecter _send_command 1 2023-07-29T14:05:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 1 -- 172.21.15.40:0/2861576484 --> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97a0076350 con 0x7f97a00737f0 2023-07-29T14:05:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 20 client.16415.objecter dump_active .. 0 homeless 2023-07-29T14:05:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 1 -- 172.21.15.40:0/2861576484 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=1004) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97a8054360 con 0x7f97c00b3550 2023-07-29T14:05:52.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.586+0000 7f97b77fe640 10 monclient: handle_get_version_reply finishing 1 version 1004 2023-07-29T14:05:52.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.587+0000 7f97bf7fe640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] conn(0x7f97a00737f0 0x7f97a0075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:52.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.587+0000 7f97bf7fe640 10 monclient: get_auth_request con 0x7f97a00737f0 auth_method 0 2023-07-29T14:05:52.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.591+0000 7f97b77fe640 10 client.16415.objecter ms_handle_connect 0x7f97a00737f0 2023-07-29T14:05:52.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.591+0000 7f97bf7fe640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] conn(0x7f97a00737f0 0x7f97a0075c80 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:52.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.615+0000 7f97b77fe640 1 -- 172.21.15.40:0/2861576484 <== osd.5 v2:172.21.15.45:6812/85181 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f97a0076350 con 0x7f97a00737f0 2023-07-29T14:05:52.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.615+0000 7f97b77fe640 10 client.16415.objecter ms_dispatch 0x7f97c0000b90 command_reply(tid 1: 0 ) v1 2023-07-29T14:05:52.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.615+0000 7f97b77fe640 10 client.16415.objecter _finish_command 1 = system:0 2023-07-29T14:05:52.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.615+0000 7f97b77fe640 20 client.16415.objecter put_session s=0x7f97a0073120 osd=5 4 2023-07-29T14:05:52.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.615+0000 7f97b77fe640 15 client.16415.objecter _session_command_op_remove 5 1 2023-07-29T14:05:52.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.629+0000 7f97c54c7640 10 client.16415.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-07-29T14:05:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.629+0000 7f97c54c7640 15 client.16415.objecter _session_command_op_assign -1 2 2023-07-29T14:05:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.629+0000 7f97c54c7640 20 client.16415.objecter _get_session s=0x7f97a0073120 osd=5 3 2023-07-29T14:05:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.629+0000 7f97c54c7640 20 client.16415.objecter put_session s=0x7f97a0073120 osd=5 3 2023-07-29T14:05:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.629+0000 7f97c54c7640 20 client.16415.objecter _get_session s=0x7f97a0073120 osd=5 3 2023-07-29T14:05:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.629+0000 7f97c54c7640 15 client.16415.objecter _session_command_op_remove -1 2 2023-07-29T14:05:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.629+0000 7f97c54c7640 20 client.16415.objecter get_session s=0x7f97a0073120 osd=5 3 2023-07-29T14:05:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.629+0000 7f97c54c7640 15 client.16415.objecter _session_command_op_assign 5 2 2023-07-29T14:05:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.629+0000 7f97c54c7640 20 client.16415.objecter put_session s=0x7f97a0073120 osd=5 4 2023-07-29T14:05:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.629+0000 7f97c54c7640 10 client.16415.objecter _send_command 2 2023-07-29T14:05:52.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.629+0000 7f97c54c7640 1 -- 172.21.15.40:0/2861576484 --> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f97c00af150 con 0x7f97a00737f0 2023-07-29T14:05:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.644+0000 7f97b77fe640 1 -- 172.21.15.40:0/2861576484 <== osd.5 v2:172.21.15.45:6812/85181 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f97b802c990 con 0x7f97a00737f0 2023-07-29T14:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.644+0000 7f97b77fe640 10 client.16415.objecter ms_dispatch 0x7f97c0000b90 command_reply(tid 2: 0 ) v1 2023-07-29T14:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.644+0000 7f97b77fe640 10 client.16415.objecter _finish_command 2 = system:0 2023-07-29T14:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.644+0000 7f97b77fe640 20 client.16415.objecter put_session s=0x7f97a0073120 osd=5 4 2023-07-29T14:05:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.644+0000 7f97b77fe640 15 client.16415.objecter _session_command_op_remove 5 2 2023-07-29T14:05:52.646 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-29T14:05:52.647 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-29T14:05:52.647 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-29T14:05:52.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 10 librados: watch_flush enter 2023-07-29T14:05:52.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 10 librados: watch_flush exit 2023-07-29T14:05:52.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 10 client.16415.objecter close_session for osd.5 2023-07-29T14:05:52.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 1 -- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] conn(0x7f97a00737f0 msgr2=0x7f97a0075c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:6812/85181,v1:172.21.15.45:6813/85181] conn(0x7f97a00737f0 0x7f97a0075c80 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 20 client.16415.objecter put_session s=0x7f97a0073120 osd=5 1 2023-07-29T14:05:52.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 20 client.16415.objecter shutdown clearing up homeless session... 2023-07-29T14:05:52.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 10 client.16415.objecter successfully canceled tick 2023-07-29T14:05:52.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 1 -- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f97a006bfb0 msgr2=0x7f97a006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f97a006bfb0 0x7f97a006e460 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f97b0001780 tx=0x7f97b0009040 comp rx=0 tx=0).stop 2023-07-29T14:05:52.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 10 monclient: shutdown 2023-07-29T14:05:52.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:52.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 1 -- 172.21.15.40:0/2861576484 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b3550 msgr2=0x7f97c01d69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 1 --2- 172.21.15.40:0/2861576484 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f97c00b3550 0x7f97c01d69a0 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 crypto rx=0x7f97a800e8a0 tx=0x7f97a800ed60 comp rx=0 tx=0).stop 2023-07-29T14:05:52.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97bf7fe640 1 -- 172.21.15.40:0/2861576484 reap_dead start 2023-07-29T14:05:52.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 1 -- 172.21.15.40:0/2861576484 shutdown_connections 2023-07-29T14:05:52.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 1 -- 172.21.15.40:0/2861576484 >> 172.21.15.40:0/2861576484 conn(0x7f97c0065920 msgr2=0x7f97c013d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:52.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 1 -- 172.21.15.40:0/2861576484 shutdown_connections 2023-07-29T14:05:52.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 1 -- 172.21.15.40:0/2861576484 wait complete. 2023-07-29T14:05:52.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.645+0000 7f97c54c7640 1 librados: shutdown 2023-07-29T14:05:52.657 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-07-29T14:05:52.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.770+0000 7fe40f714640 10 monclient: get_monmap_and_config 2023-07-29T14:05:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.770+0000 7fe40f714640 10 monclient: build_initial_monmap 2023-07-29T14:05:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.770+0000 7fe40f714640 10 monclient: monmap: 2023-07-29T14:05:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:52.770513+0000 2023-07-29T14:05:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:52.770513+0000 2023-07-29T14:05:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:52.773 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.770+0000 7fe40f714640 1 Processor -- start 2023-07-29T14:05:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 1 -- start start 2023-07-29T14:05:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 10 monclient: init 2023-07-29T14:05:52.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:52.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2023-07-29T14:05:52.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4080b3ae0 0x7fe40813dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 10 monclient(hunting): picked mon.noname-a con 0x7fe4080b3ae0 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:52.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe4080a8910 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:52.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe408141320 0x7fe40813e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 10 monclient(hunting): picked mon.noname-c con 0x7fe408141320 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:52.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4080b3eb0 con 0x7fe4080a8910 2023-07-29T14:05:52.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:52.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4080b0f10 con 0x7fe4080b3ae0 2023-07-29T14:05:52.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:52.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4080a9950 con 0x7fe408141320 2023-07-29T14:05:52.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:52.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.771+0000 7fe40f714640 10 monclient(hunting): authenticate will time out at 3674.903863s 2023-07-29T14:05:52.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.777+0000 7fe40d4b0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4080b3ae0 0x7fe40813dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:52.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.778+0000 7fe40ccaf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe4080a8910 0x7fe408140c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38660/0 (socket says 172.21.15.40:38660) 2023-07-29T14:05:52.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.778+0000 7fe40ccaf640 1 -- 172.21.15.40:0/1507064639 learned_addr learned my addr 172.21.15.40:0/1507064639 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:52.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.779+0000 7fe40d4b0640 10 monclient(hunting): get_auth_request con 0x7fe4080b3ae0 auth_method 0 2023-07-29T14:05:52.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.779+0000 7fe40d4b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:52.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.779+0000 7fe40d4b0640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:52.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.779+0000 7fe40d4b0640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:52.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.781+0000 7fe40d4b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:52.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.781+0000 7fe40d4b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.781+0000 7fe40d4b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:52.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.784+0000 7fe40d4b0640 10 monclient(hunting): handle_auth_done global_id 14976 payload 274 2023-07-29T14:05:52.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.784+0000 7fe40d4b0640 1 -- 172.21.15.40:0/1507064639 >> v2:172.21.15.45:3300/0 conn(0x7fe408141320 msgr2=0x7fe40813e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:52.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.784+0000 7fe40d4b0640 1 --2- 172.21.15.40:0/1507064639 >> v2:172.21.15.45:3300/0 conn(0x7fe408141320 0x7fe40813e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.784+0000 7fe40d4b0640 1 -- 172.21.15.40:0/1507064639 >> v1:172.21.15.40:6789/0 conn(0x7fe4080a8910 legacy=0x7fe408140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.784+0000 7fe40d4b0640 10 monclient: _finish_hunting 0 2023-07-29T14:05:52.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.784+0000 7fe40d4b0640 1 monclient: found mon.noname-a 2023-07-29T14:05:52.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.784+0000 7fe40d4b0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:52.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.784+0000 7fe40d4b0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T14:05:52.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.784+0000 7fe40d4b0640 1 -- 172.21.15.40:0/1507064639 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4080a9c30 con 0x7fe4080b3ae0 2023-07-29T14:05:52.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.787+0000 7fe40d4b0640 1 --2- 172.21.15.40:0/1507064639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4080b3ae0 0x7fe40813dc90 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 crypto rx=0x7fe408141ce0 tx=0x7fe3f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94a642920c4fbc0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:52.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 1 -- 172.21.15.40:0/1507064639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f802e070 con 0x7fe4080b3ae0 2023-07-29T14:05:52.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:52.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T14:05:52.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 10 monclient: dump: 2023-07-29T14:05:52.799 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:52.799 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:52.799 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.800 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.800 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:52.800 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:52.802 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:52.802 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:52.802 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 10 monclient: _finish_auth 0 2023-07-29T14:05:52.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 10 monclient: _check_auth_tickets 2023-07-29T14:05:52.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:52.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 1 -- 172.21.15.40:0/1507064639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f80315a0 con 0x7fe4080b3ae0 2023-07-29T14:05:52.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe40f714640 5 monclient: authenticate success, global_id 14976 2023-07-29T14:05:52.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe40f714640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T14:05:52.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:52.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe40f714640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:52.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe40f714640 4 set_mon_vals no callback set 2023-07-29T14:05:52.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 1 -- 172.21.15.40:0/1507064639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8031930 con 0x7fe4080b3ae0 2023-07-29T14:05:52.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:52.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:52.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.793+0000 7fe3fffff640 10 monclient: dump: 2023-07-29T14:05:52.806 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:52.806 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:52.807 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.807 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.807 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:52.807 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.807 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:52.808 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:52.825 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.794+0000 7fe40f714640 10 monclient: shutdown 2023-07-29T14:05:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.794+0000 7fe40f714640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.794+0000 7fe40f714640 1 -- 172.21.15.40:0/1507064639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4080b3ae0 msgr2=0x7fe40813dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.794+0000 7fe40f714640 1 --2- 172.21.15.40:0/1507064639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4080b3ae0 0x7fe40813dc90 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 crypto rx=0x7fe408141ce0 tx=0x7fe3f802b840 comp rx=0 tx=0).stop 2023-07-29T14:05:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.794+0000 7fe40f714640 1 -- 172.21.15.40:0/1507064639 shutdown_connections 2023-07-29T14:05:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.794+0000 7fe40f714640 1 --2- 172.21.15.40:0/1507064639 >> v2:172.21.15.45:3300/0 conn(0x7fe408141320 0x7fe40813e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.794+0000 7fe40f714640 1 --2- 172.21.15.40:0/1507064639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4080b3ae0 0x7fe40813dc90 unknown :-1 s=CLOSED pgs=3221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.794+0000 7fe40f714640 1 -- 172.21.15.40:0/1507064639 >> 172.21.15.40:0/1507064639 conn(0x7fe408070fc0 msgr2=0x7fe4080713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.794+0000 7fe40f714640 1 -- 172.21.15.40:0/1507064639 shutdown_connections 2023-07-29T14:05:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.794+0000 7fe40f714640 1 -- 172.21.15.40:0/1507064639 wait complete. 2023-07-29T14:05:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.795+0000 7fe40f714640 10 monclient: build_initial_monmap 2023-07-29T14:05:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.795+0000 7fe40f714640 10 monclient: monmap: 2023-07-29T14:05:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:52.829 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.795+0000 7fe40f714640 1 librados: starting msgr at 2023-07-29T14:05:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.795+0000 7fe40f714640 1 librados: starting objecter 2023-07-29T14:05:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.795+0000 7fe40f714640 1 Processor -- start 2023-07-29T14:05:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.795+0000 7fe40f714640 1 -- start start 2023-07-29T14:05:52.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.795+0000 7fe40f714640 1 librados: setting wanted keys 2023-07-29T14:05:52.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.795+0000 7fe40f714640 1 librados: calling monclient init 2023-07-29T14:05:52.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.795+0000 7fe40f714640 10 monclient: init 2023-07-29T14:05:52.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:52.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T14:05:52.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4080a8910 0x7fe4081d92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 10 monclient(hunting): picked mon.noname-a con 0x7fe4080a8910 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:52.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4080b3ae0 0x7fe40813bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 10 monclient(hunting): picked mon.noname-c con 0x7fe4080b3ae0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:52.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe408141320 0x7fe40813c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 10 monclient(hunting): picked mon.noname-b con 0x7fe408141320 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080ae9f0 con 0x7fe4080a8910 2023-07-29T14:05:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40813c950 con 0x7fe408141320 2023-07-29T14:05:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:52.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080ae580 con 0x7fe4080b3ae0 2023-07-29T14:05:52.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:52.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.796+0000 7fe40f714640 10 monclient(hunting): authenticate will time out at 3674.928540s 2023-07-29T14:05:52.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.797+0000 7fe40d4b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4080a8910 0x7fe4081d92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:52.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.797+0000 7fe40d4b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4080a8910 0x7fe4081d92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60268/0 (socket says 172.21.15.40:60268) 2023-07-29T14:05:52.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.797+0000 7fe40d4b0640 1 -- 172.21.15.40:0/638806064 learned_addr learned my addr 172.21.15.40:0/638806064 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:52.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.797+0000 7fe40d4b0640 10 monclient(hunting): get_auth_request con 0x7fe4080a8910 auth_method 0 2023-07-29T14:05:52.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.797+0000 7fe40d4b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:52.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.797+0000 7fe40d4b0640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:52.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.797+0000 7fe40d4b0640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:52.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.798+0000 7fe40d4b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:52.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.798+0000 7fe40d4b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:52.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.798+0000 7fe40d4b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:52.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.801+0000 7fe40d4b0640 10 monclient(hunting): handle_auth_done global_id 14979 payload 824 2023-07-29T14:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.801+0000 7fe40d4b0640 1 -- 172.21.15.40:0/638806064 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4080b3ae0 msgr2=0x7fe40813bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.801+0000 7fe40d4b0640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4080b3ae0 0x7fe40813bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.801+0000 7fe40d4b0640 1 -- 172.21.15.40:0/638806064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe408141320 msgr2=0x7fe40813c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.801+0000 7fe40d4b0640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe408141320 0x7fe40813c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.801+0000 7fe40d4b0640 10 monclient: _finish_hunting 0 2023-07-29T14:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.801+0000 7fe40d4b0640 1 monclient: found mon.noname-a 2023-07-29T14:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.801+0000 7fe40d4b0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:52.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.801+0000 7fe40d4b0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T14:05:52.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.801+0000 7fe40d4b0640 1 -- 172.21.15.40:0/638806064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8009400 con 0x7fe4080a8910 2023-07-29T14:05:52.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.804+0000 7fe40d4b0640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4080a8910 0x7fe4081d92f0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009890 tx=0x7fe3f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.804+0000 7fe3fdffb640 10 client.?.objecter ms_handle_connect 0x7fe4080a8910 2023-07-29T14:05:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.804+0000 7fe3fdffb640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3fdffb640 1 -- 172.21.15.40:0/638806064 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8032980 con 0x7fe4080a8910 2023-07-29T14:05:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3fdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3fdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T14:05:52.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3fdffb640 10 monclient: dump: 2023-07-29T14:05:52.855 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:52.855 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:52.855 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.855 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.855 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:52.855 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.856 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:52.856 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:52.856 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:52.856 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3fdffb640 10 monclient: _finish_auth 0 2023-07-29T14:05:52.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3fdffb640 10 monclient: _check_auth_tickets 2023-07-29T14:05:52.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:52.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3fdffb640 1 -- 172.21.15.40:0/638806064 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f8032ae0 con 0x7fe4080a8910 2023-07-29T14:05:52.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe40f714640 5 monclient: authenticate success, global_id 14979 2023-07-29T14:05:52.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3fdffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:52.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe40f714640 10 monclient: _renew_subs 2023-07-29T14:05:52.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3fdffb640 1 -- 172.21.15.40:0/638806064 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8032e90 con 0x7fe4080a8910 2023-07-29T14:05:52.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe40f714640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:52.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe40f714640 1 -- 172.21.15.40:0/638806064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40813cb50 con 0x7fe4080a8910 2023-07-29T14:05:52.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3fdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:52.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.808+0000 7fe3fdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:52.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.807+0000 7fe3ff7fe640 4 set_mon_vals no callback set 2023-07-29T14:05:52.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.808+0000 7fe3fdffb640 10 monclient: dump: 2023-07-29T14:05:52.859 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:52.859 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:52.859 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.859 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:52.859 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:52.859 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:52.859 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:52.860 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:52.860 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:52.861 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.808+0000 7fe40f714640 10 client.14979.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.808+0000 7fe40f714640 10 monclient: _renew_subs 2023-07-29T14:05:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.808+0000 7fe40f714640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:52.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.808+0000 7fe40f714640 1 -- 172.21.15.40:0/638806064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4081d9ee0 con 0x7fe4080a8910 2023-07-29T14:05:52.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.808+0000 7fe40f714640 1 librados: init done 2023-07-29T14:05:52.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.809+0000 7fe40f714640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:52.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.809+0000 7fe40f714640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:52.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.809+0000 7fe40f714640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.40:3300/0 2023-07-29T14:05:52.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.809+0000 7fe40f714640 1 -- 172.21.15.40:0/638806064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4080b02e0 con 0x7fe4080a8910 2023-07-29T14:05:52.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.841+0000 7fe3fdffb640 1 -- 172.21.15.40:0/638806064 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fe3f80453f0 con 0x7fe4080a8910 2023-07-29T14:05:52.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.841+0000 7fe3fdffb640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe3ec06bfb0 0x7fe3ec06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:52.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.842+0000 7fe40ccaf640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe3ec06bfb0 0x7fe3ec06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:52.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.842+0000 7fe40ccaf640 10 monclient: get_auth_request con 0x7fe3ec06bfb0 auth_method 0 2023-07-29T14:05:52.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.842+0000 7fe40ccaf640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe3ec06bfb0 0x7fe3ec06e460 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe404000f60 tx=0x7fe404007a80 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:52.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:52.842+0000 7fe3fdffb640 10 client.14979.objecter ms_handle_connect 0x7fe3ec06bfb0 2023-07-29T14:05:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.100+0000 7fe3fdffb640 1 -- 172.21.15.40:0/638806064 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7fe3f807e050 con 0x7fe4080a8910 2023-07-29T14:05:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.100+0000 7fe3fdffb640 10 client.14979.objecter ms_dispatch 0x7fe408000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.100+0000 7fe3fdffb640 3 client.14979.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.100+0000 7fe3fdffb640 3 client.14979.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.100+0000 7fe3fdffb640 20 client.14979.objecter dump_active .. 0 homeless 2023-07-29T14:05:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.100+0000 7fe3fdffb640 1 -- 172.21.15.40:0/638806064 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7fe3f80763d0 con 0x7fe4080a8910 2023-07-29T14:05:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.100+0000 7fe3fdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.100+0000 7fe3fdffb640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:53.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.278+0000 7fe40f714640 1 -- 172.21.15.40:0/638806064 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4080b29f0 con 0x7fe3ec06bfb0 2023-07-29T14:05:53.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.278+0000 7fe3fdffb640 1 -- 172.21.15.40:0/638806064 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe4040058b0 con 0x7fe3ec06bfb0 2023-07-29T14:05:53.280 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2023-07-29T14:05:53.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.280+0000 7fe40f714640 10 librados: watch_flush enter 2023-07-29T14:05:53.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.280+0000 7fe40f714640 10 librados: watch_flush exit 2023-07-29T14:05:53.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.280+0000 7fe40f714640 20 client.14979.objecter shutdown clearing up homeless session... 2023-07-29T14:05:53.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.280+0000 7fe40f714640 10 client.14979.objecter successfully canceled tick 2023-07-29T14:05:53.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 -- 172.21.15.40:0/638806064 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe3ec06bfb0 msgr2=0x7fe3ec06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe3ec06bfb0 0x7fe3ec06e460 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe404000f60 tx=0x7fe404007a80 comp rx=0 tx=0).stop 2023-07-29T14:05:53.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 10 monclient: shutdown 2023-07-29T14:05:53.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:53.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 -- 172.21.15.40:0/638806064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4080a8910 msgr2=0x7fe4081d92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4080a8910 0x7fe4081d92f0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009890 tx=0x7fe3f80046c0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 -- 172.21.15.40:0/638806064 shutdown_connections 2023-07-29T14:05:53.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe3ec06bfb0 0x7fe3ec06e460 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe408141320 0x7fe40813c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4080b3ae0 0x7fe40813bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 --2- 172.21.15.40:0/638806064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4080a8910 0x7fe4081d92f0 unknown :-1 s=CLOSED pgs=3222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 -- 172.21.15.40:0/638806064 >> 172.21.15.40:0/638806064 conn(0x7fe408070fc0 msgr2=0x7fe408071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:53.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 -- 172.21.15.40:0/638806064 shutdown_connections 2023-07-29T14:05:53.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 -- 172.21.15.40:0/638806064 wait complete. 2023-07-29T14:05:53.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.281+0000 7fe40f714640 1 librados: shutdown 2023-07-29T14:05:53.294 INFO:tasks.ceph:Scrubbing osd.6 2023-07-29T14:05:53.294 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-07-29T14:05:53.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.409+0000 7f8cf26bd640 10 monclient: get_monmap_and_config 2023-07-29T14:05:53.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.409+0000 7f8cf26bd640 10 monclient: build_initial_monmap 2023-07-29T14:05:53.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.409+0000 7f8cf26bd640 10 monclient: monmap: 2023-07-29T14:05:53.411 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:53.411 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:53.411 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:53.409437+0000 2023-07-29T14:05:53.411 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:53.409437+0000 2023-07-29T14:05:53.411 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:53.411 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.412 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:53.412 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:53.412 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:53.412 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:53.412 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:53.412 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.409+0000 7f8cf26bd640 1 Processor -- start 2023-07-29T14:05:53.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 1 -- start start 2023-07-29T14:05:53.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 10 monclient: init 2023-07-29T14:05:53.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:53.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2023-07-29T14:05:53.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8cec0a86c0 0x7f8cec140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 10 monclient(hunting): picked mon.noname-a con 0x7f8cec0a86c0 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:53.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8cec1410e0 0x7f8cec13dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 10 monclient(hunting): picked mon.noname-c con 0x7f8cec1410e0 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:53.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 0x7f8cec13e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 10 monclient(hunting): picked mon.noname-b con 0x7f8cec0b2f60 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:53.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:53.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cec0a8a90 con 0x7f8cec0a86c0 2023-07-29T14:05:53.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:53.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cec0a98e0 con 0x7f8cec1410e0 2023-07-29T14:05:53.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:53.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec0a9bc0 con 0x7f8cec0b2f60 2023-07-29T14:05:53.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:53.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.410+0000 7f8cf26bd640 10 monclient(hunting): authenticate will time out at 3675.542811s 2023-07-29T14:05:53.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cf0c5a640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 0x7f8cec13e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cebfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8cec0a86c0 0x7f8cec140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8ceb7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8cec1410e0 0x7f8cec13dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cf0c5a640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 0x7f8cec13e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56700/0 (socket says 172.21.15.40:56700) 2023-07-29T14:05:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cf0c5a640 1 -- 172.21.15.40:0/3120272947 learned_addr learned my addr 172.21.15.40:0/3120272947 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cf0c5a640 10 monclient(hunting): get_auth_request con 0x7f8cec0b2f60 auth_method 0 2023-07-29T14:05:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cf0c5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cf0c5a640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cf0c5a640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cebfff640 10 monclient(hunting): get_auth_request con 0x7f8cec0a86c0 auth_method 0 2023-07-29T14:05:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cebfff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8cebfff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8ceb7fe640 10 monclient(hunting): get_auth_request con 0x7f8cec1410e0 auth_method 0 2023-07-29T14:05:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8ceb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8ceb7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.411+0000 7f8ceb7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.412+0000 7f8cf0c5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.412+0000 7f8cf0c5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.412+0000 7f8cf0c5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.412+0000 7f8cebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.412+0000 7f8cebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.412+0000 7f8cebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.412+0000 7f8ceb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.412+0000 7f8ceb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.412+0000 7f8ceb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.413+0000 7f8cf0c5a640 10 monclient(hunting): handle_auth_done global_id 16418 payload 274 2023-07-29T14:05:53.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.413+0000 7f8cf0c5a640 1 -- 172.21.15.40:0/3120272947 >> v2:172.21.15.45:3300/0 conn(0x7f8cec1410e0 msgr2=0x7f8cec13dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:53.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.413+0000 7f8cf0c5a640 1 --2- 172.21.15.40:0/3120272947 >> v2:172.21.15.45:3300/0 conn(0x7f8cec1410e0 0x7f8cec13dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.413+0000 7f8cf0c5a640 1 -- 172.21.15.40:0/3120272947 >> v2:172.21.15.40:3300/0 conn(0x7f8cec0a86c0 msgr2=0x7f8cec140ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:53.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.413+0000 7f8cf0c5a640 1 --2- 172.21.15.40:0/3120272947 >> v2:172.21.15.40:3300/0 conn(0x7f8cec0a86c0 0x7f8cec140ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.413+0000 7f8cf0c5a640 10 monclient: _finish_hunting 0 2023-07-29T14:05:53.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.413+0000 7f8cf0c5a640 1 monclient: found mon.noname-b 2023-07-29T14:05:53.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.413+0000 7f8cf0c5a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:53.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.413+0000 7f8cf0c5a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:53.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.413+0000 7f8cf0c5a640 1 -- 172.21.15.40:0/3120272947 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec13e6e0 con 0x7f8cec0b2f60 2023-07-29T14:05:53.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.415+0000 7f8cf0c5a640 1 --2- 172.21.15.40:0/3120272947 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 0x7f8cec13e110 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0a91d0 tx=0x7f8ce402be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcd09387cfcd12c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:53.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 1 -- 172.21.15.40:0/3120272947 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce402c9e0 con 0x7f8cec0b2f60 2023-07-29T14:05:53.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:53.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:53.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 10 monclient: dump: 2023-07-29T14:05:53.438 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:53.438 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:53.438 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.438 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.438 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:53.438 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.439 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:53.439 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:53.439 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:53.439 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 10 monclient: _finish_auth 0 2023-07-29T14:05:53.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:53.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:53.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 1 -- 172.21.15.40:0/3120272947 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce402cb40 con 0x7f8cec0b2f60 2023-07-29T14:05:53.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:53.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 1 -- 172.21.15.40:0/3120272947 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40343f0 con 0x7f8cec0b2f60 2023-07-29T14:05:53.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:53.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:53.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8ceaffd640 10 monclient: dump: 2023-07-29T14:05:53.443 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:53.443 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:53.443 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.443 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.443 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:53.443 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.444 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:53.444 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:53.444 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:53.450 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8cf26bd640 5 monclient: authenticate success, global_id 16418 2023-07-29T14:05:53.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8cf26bd640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:53.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8cf26bd640 4 set_mon_vals no callback set 2023-07-29T14:05:53.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8cf26bd640 10 monclient: shutdown 2023-07-29T14:05:53.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8cf26bd640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8cf26bd640 1 -- 172.21.15.40:0/3120272947 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 msgr2=0x7f8cec13e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.419+0000 7f8cf26bd640 1 --2- 172.21.15.40:0/3120272947 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 0x7f8cec13e110 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0a91d0 tx=0x7f8ce402be40 comp rx=0 tx=0).stop 2023-07-29T14:05:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.420+0000 7f8cf26bd640 1 -- 172.21.15.40:0/3120272947 shutdown_connections 2023-07-29T14:05:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.420+0000 7f8cf26bd640 1 --2- 172.21.15.40:0/3120272947 >> v2:172.21.15.45:3300/0 conn(0x7f8cec1410e0 0x7f8cec13dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.420+0000 7f8cf26bd640 1 --2- 172.21.15.40:0/3120272947 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 0x7f8cec13e110 unknown :-1 s=CLOSED pgs=3237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.420+0000 7f8cf26bd640 1 --2- 172.21.15.40:0/3120272947 >> v2:172.21.15.40:3300/0 conn(0x7f8cec0a86c0 0x7f8cec140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.420+0000 7f8cf26bd640 1 -- 172.21.15.40:0/3120272947 >> 172.21.15.40:0/3120272947 conn(0x7f8cec065920 msgr2=0x7f8cec065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.420+0000 7f8cf26bd640 1 -- 172.21.15.40:0/3120272947 shutdown_connections 2023-07-29T14:05:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.420+0000 7f8cf26bd640 1 -- 172.21.15.40:0/3120272947 wait complete. 2023-07-29T14:05:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.420+0000 7f8cf26bd640 10 monclient: build_initial_monmap 2023-07-29T14:05:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.420+0000 7f8cf26bd640 10 monclient: monmap: 2023-07-29T14:05:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:53.455 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 librados: starting msgr at 2023-07-29T14:05:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 librados: starting objecter 2023-07-29T14:05:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 Processor -- start 2023-07-29T14:05:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 -- start start 2023-07-29T14:05:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 librados: setting wanted keys 2023-07-29T14:05:53.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 librados: calling monclient init 2023-07-29T14:05:53.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 10 monclient: init 2023-07-29T14:05:53.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:53.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 10 monclient: _add_conns ranks=[1,2,0] 2023-07-29T14:05:53.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec0a86c0 0x7f8cec13fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 10 monclient(hunting): picked mon.noname-b con 0x7f8cec0a86c0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:53.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 0x7f8cec143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 10 monclient(hunting): picked mon.noname-c con 0x7f8cec0b2f60 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cec1410e0 0x7f8cec1362d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 10 monclient(hunting): picked mon.noname-a con 0x7f8cec1410e0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec140960 con 0x7f8cec1410e0 2023-07-29T14:05:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec1446a0 con 0x7f8cec0a86c0 2023-07-29T14:05:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec144160 con 0x7f8cec0b2f60 2023-07-29T14:05:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.421+0000 7f8cf26bd640 10 monclient(hunting): authenticate will time out at 3675.553989s 2023-07-29T14:05:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8ceb7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 0x7f8cec143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8cf0c5a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cec1410e0 0x7f8cec1362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8ceb7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 0x7f8cec143c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56704/0 (socket says 172.21.15.40:56704) 2023-07-29T14:05:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8ceb7fe640 1 -- 172.21.15.40:0/2453977447 learned_addr learned my addr 172.21.15.40:0/2453977447 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8ceb7fe640 10 monclient(hunting): get_auth_request con 0x7f8cec0b2f60 auth_method 0 2023-07-29T14:05:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8ceb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8ceb7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8ceb7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8cebfff640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec0a86c0 0x7f8cec13fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8cf0c5a640 10 monclient(hunting): get_auth_request con 0x7f8cec1410e0 auth_method 0 2023-07-29T14:05:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8cf0c5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8cf0c5a640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.422+0000 7f8cf0c5a640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:53.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8ceb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:53.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8ceb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:53.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8ceb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8cebfff640 10 monclient(hunting): get_auth_request con 0x7f8cec0a86c0 auth_method 0 2023-07-29T14:05:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8cebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8cebfff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8cebfff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8cf0c5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8cf0c5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8cf0c5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8cebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8cebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.423+0000 7f8cebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.425+0000 7f8ceb7fe640 10 monclient(hunting): handle_auth_done global_id 16421 payload 824 2023-07-29T14:05:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.425+0000 7f8ceb7fe640 1 -- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec0a86c0 msgr2=0x7f8cec13fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.425+0000 7f8ceb7fe640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cec0a86c0 0x7f8cec13fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.425+0000 7f8ceb7fe640 1 -- 172.21.15.40:0/2453977447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cec1410e0 msgr2=0x7f8cec1362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.425+0000 7f8ceb7fe640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cec1410e0 0x7f8cec1362d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.425+0000 7f8ceb7fe640 10 monclient: _finish_hunting 0 2023-07-29T14:05:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.425+0000 7f8ceb7fe640 1 monclient: found mon.noname-c 2023-07-29T14:05:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.425+0000 7f8ceb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.425+0000 7f8ceb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.425+0000 7f8ceb7fe640 1 -- 172.21.15.40:0/2453977447 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce4009b30 con 0x7f8cec0b2f60 2023-07-29T14:05:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.428+0000 7f8ceb7fe640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 0x7f8cec143c20 secure :-1 s=READY pgs=3238 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000e870 tx=0x7f8ce000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.428+0000 7f8ce8ff9640 10 client.?.objecter ms_handle_connect 0x7f8cec0b2f60 2023-07-29T14:05:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.428+0000 7f8ce8ff9640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 1 -- 172.21.15.40:0/2453977447 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00094f0 con 0x7f8cec0b2f60 2023-07-29T14:05:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 10 monclient: dump: 2023-07-29T14:05:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:53.477 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 10 monclient: _finish_auth 0 2023-07-29T14:05:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 10 monclient: _check_auth_tickets 2023-07-29T14:05:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 1 -- 172.21.15.40:0/2453977447 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce000c940 con 0x7f8cec0b2f60 2023-07-29T14:05:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 1 -- 172.21.15.40:0/2453977447 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000ccf0 con 0x7f8cec0b2f60 2023-07-29T14:05:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8ce8ff9640 10 monclient: dump: 2023-07-29T14:05:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:53.480 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8cf26bd640 5 monclient: authenticate success, global_id 16421 2023-07-29T14:05:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8cf26bd640 10 monclient: _renew_subs 2023-07-29T14:05:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8cf26bd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8cf26bd640 1 -- 172.21.15.40:0/2453977447 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cec1443e0 con 0x7f8cec0b2f60 2023-07-29T14:05:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8cea7fc640 4 set_mon_vals no callback set 2023-07-29T14:05:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8cf26bd640 10 client.16421.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8cf26bd640 10 monclient: _renew_subs 2023-07-29T14:05:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8cf26bd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8cf26bd640 1 -- 172.21.15.40:0/2453977447 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cec1369f0 con 0x7f8cec0b2f60 2023-07-29T14:05:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.431+0000 7f8cf26bd640 1 librados: init done 2023-07-29T14:05:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.432+0000 7f8cf26bd640 10 client.16421.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.432+0000 7f8cf26bd640 15 client.16421.objecter _session_command_op_assign -1 1 2023-07-29T14:05:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.432+0000 7f8cf26bd640 20 client.16421.objecter _get_session osd=-1 returning homeless 2023-07-29T14:05:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.432+0000 7f8cf26bd640 10 client.16421.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.432+0000 7f8cf26bd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.432+0000 7f8cf26bd640 1 -- 172.21.15.40:0/2453977447 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8cec0b2a40 con 0x7f8cec0b2f60 2023-07-29T14:05:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.451+0000 7f8ce8ff9640 1 -- 172.21.15.40:0/2453977447 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f8ce0013470 con 0x7f8cec0b2f60 2023-07-29T14:05:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.451+0000 7f8ce8ff9640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f8cd806bfb0 0x7f8cd806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.451+0000 7f8cebfff640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f8cd806bfb0 0x7f8cd806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.452+0000 7f8cebfff640 10 monclient: get_auth_request con 0x7f8cd806bfb0 auth_method 0 2023-07-29T14:05:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.452+0000 7f8ce8ff9640 10 client.16421.objecter ms_handle_connect 0x7f8cd806bfb0 2023-07-29T14:05:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.452+0000 7f8cebfff640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f8cd806bfb0 0x7f8cd806e460 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009b40 tx=0x7f8cdc009190 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 1 -- 172.21.15.40:0/2453977447 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f8ce005be80 con 0x7f8cec0b2f60 2023-07-29T14:05:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 10 client.16421.objecter ms_dispatch 0x7f8cec000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 3 client.16421.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 3 client.16421.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 10 client.16421.objecter checking command 1 2023-07-29T14:05:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] conn(0x7f8cd80737f0 0x7f8cd8075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 20 client.16421.objecter _get_session s=0x7f8cd8073120 osd=6 3 2023-07-29T14:05:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 20 client.16421.objecter put_session s=0x7f8cd8073120 osd=6 3 2023-07-29T14:05:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 15 client.16421.objecter _session_command_op_remove -1 1 2023-07-29T14:05:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 20 client.16421.objecter _get_session s=0x7f8cd8073120 osd=6 3 2023-07-29T14:05:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 20 client.16421.objecter get_session s=0x7f8cd8073120 osd=6 3 2023-07-29T14:05:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 15 client.16421.objecter _session_command_op_assign 6 1 2023-07-29T14:05:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 20 client.16421.objecter put_session s=0x7f8cd8073120 osd=6 4 2023-07-29T14:05:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 10 client.16421.objecter _send_command 1 2023-07-29T14:05:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 1 -- 172.21.15.40:0/2453977447 --> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cd8076350 con 0x7f8cd80737f0 2023-07-29T14:05:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 20 client.16421.objecter dump_active .. 0 homeless 2023-07-29T14:05:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 1 -- 172.21.15.40:0/2453977447 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=1004) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ce00542b0 con 0x7f8cec0b2f60 2023-07-29T14:05:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.454+0000 7f8ce8ff9640 10 monclient: handle_get_version_reply finishing 1 version 1004 2023-07-29T14:05:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.455+0000 7f8cf0c5a640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] conn(0x7f8cd80737f0 0x7f8cd8075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.456+0000 7f8cf0c5a640 10 monclient: get_auth_request con 0x7f8cd80737f0 auth_method 0 2023-07-29T14:05:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.459+0000 7f8cf0c5a640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] conn(0x7f8cd80737f0 0x7f8cd8075c80 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.459+0000 7f8ce8ff9640 10 client.16421.objecter ms_handle_connect 0x7f8cd80737f0 2023-07-29T14:05:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.485+0000 7f8ce8ff9640 1 -- 172.21.15.40:0/2453977447 <== osd.6 v2:172.21.15.45:6810/85184 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8cd8076350 con 0x7f8cd80737f0 2023-07-29T14:05:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.485+0000 7f8ce8ff9640 10 client.16421.objecter ms_dispatch 0x7f8cec000b90 command_reply(tid 1: 0 ) v1 2023-07-29T14:05:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.485+0000 7f8ce8ff9640 10 client.16421.objecter _finish_command 1 = system:0 2023-07-29T14:05:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.485+0000 7f8ce8ff9640 20 client.16421.objecter put_session s=0x7f8cd8073120 osd=6 4 2023-07-29T14:05:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.485+0000 7f8ce8ff9640 15 client.16421.objecter _session_command_op_remove 6 1 2023-07-29T14:05:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.497+0000 7f8cf26bd640 10 client.16421.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-07-29T14:05:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.497+0000 7f8cf26bd640 15 client.16421.objecter _session_command_op_assign -1 2 2023-07-29T14:05:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.497+0000 7f8cf26bd640 20 client.16421.objecter _get_session s=0x7f8cd8073120 osd=6 3 2023-07-29T14:05:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.497+0000 7f8cf26bd640 20 client.16421.objecter put_session s=0x7f8cd8073120 osd=6 3 2023-07-29T14:05:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.497+0000 7f8cf26bd640 20 client.16421.objecter _get_session s=0x7f8cd8073120 osd=6 3 2023-07-29T14:05:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.497+0000 7f8cf26bd640 15 client.16421.objecter _session_command_op_remove -1 2 2023-07-29T14:05:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.497+0000 7f8cf26bd640 20 client.16421.objecter get_session s=0x7f8cd8073120 osd=6 3 2023-07-29T14:05:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.497+0000 7f8cf26bd640 15 client.16421.objecter _session_command_op_assign 6 2 2023-07-29T14:05:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.497+0000 7f8cf26bd640 20 client.16421.objecter put_session s=0x7f8cd8073120 osd=6 4 2023-07-29T14:05:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.497+0000 7f8cf26bd640 10 client.16421.objecter _send_command 2 2023-07-29T14:05:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.497+0000 7f8cf26bd640 1 -- 172.21.15.40:0/2453977447 --> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8cec136c70 con 0x7f8cd80737f0 2023-07-29T14:05:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8ce8ff9640 1 -- 172.21.15.40:0/2453977447 <== osd.6 v2:172.21.15.45:6810/85184 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8cec136c70 con 0x7f8cd80737f0 2023-07-29T14:05:53.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8ce8ff9640 10 client.16421.objecter ms_dispatch 0x7f8cec000b90 command_reply(tid 2: 0 ) v1 2023-07-29T14:05:53.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8ce8ff9640 10 client.16421.objecter _finish_command 2 = system:0 2023-07-29T14:05:53.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8ce8ff9640 20 client.16421.objecter put_session s=0x7f8cd8073120 osd=6 4 2023-07-29T14:05:53.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8ce8ff9640 15 client.16421.objecter _session_command_op_remove 6 2 2023-07-29T14:05:53.514 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-29T14:05:53.514 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-29T14:05:53.514 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-29T14:05:53.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 10 librados: watch_flush enter 2023-07-29T14:05:53.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 10 librados: watch_flush exit 2023-07-29T14:05:53.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 10 client.16421.objecter close_session for osd.6 2023-07-29T14:05:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 1 -- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] conn(0x7f8cd80737f0 msgr2=0x7f8cd8075c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] conn(0x7f8cd80737f0 0x7f8cd8075c80 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 20 client.16421.objecter put_session s=0x7f8cd8073120 osd=6 1 2023-07-29T14:05:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 20 client.16421.objecter shutdown clearing up homeless session... 2023-07-29T14:05:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 10 client.16421.objecter successfully canceled tick 2023-07-29T14:05:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 1 -- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f8cd806bfb0 msgr2=0x7f8cd806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f8cd806bfb0 0x7f8cd806e460 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009b40 tx=0x7f8cdc009190 comp rx=0 tx=0).stop 2023-07-29T14:05:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 10 monclient: shutdown 2023-07-29T14:05:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:53.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 1 -- 172.21.15.40:0/2453977447 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 msgr2=0x7f8cec143c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 1 --2- 172.21.15.40:0/2453977447 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8cec0b2f60 0x7f8cec143c20 secure :-1 s=READY pgs=3238 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000e870 tx=0x7f8ce000ed30 comp rx=0 tx=0).stop 2023-07-29T14:05:53.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf0c5a640 1 -- 172.21.15.40:0/2453977447 reap_dead start 2023-07-29T14:05:53.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 1 -- 172.21.15.40:0/2453977447 shutdown_connections 2023-07-29T14:05:53.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 1 -- 172.21.15.40:0/2453977447 >> 172.21.15.40:0/2453977447 conn(0x7f8cec065920 msgr2=0x7f8cec13c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:53.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 1 -- 172.21.15.40:0/2453977447 shutdown_connections 2023-07-29T14:05:53.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 1 -- 172.21.15.40:0/2453977447 wait complete. 2023-07-29T14:05:53.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.513+0000 7f8cf26bd640 1 librados: shutdown 2023-07-29T14:05:53.524 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-07-29T14:05:53.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.633+0000 7f3b0af2b640 10 monclient: get_monmap_and_config 2023-07-29T14:05:53.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.633+0000 7f3b0af2b640 10 monclient: build_initial_monmap 2023-07-29T14:05:53.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.633+0000 7f3b0af2b640 10 monclient: monmap: 2023-07-29T14:05:53.642 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:53.642 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:53.642 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:53.634051+0000 2023-07-29T14:05:53.642 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:53.634051+0000 2023-07-29T14:05:53.643 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:53.643 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.643 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:53.643 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:53.643 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:53.643 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:53.643 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:53.644 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.634+0000 7f3b0af2b640 1 Processor -- start 2023-07-29T14:05:53.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.634+0000 7f3b0af2b640 1 -- start start 2023-07-29T14:05:53.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.634+0000 7f3b0af2b640 10 monclient: init 2023-07-29T14:05:53.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:53.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 10 monclient: _add_conns ranks=[1,3,0,2,4] 2023-07-29T14:05:53.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3b040b74a0 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:53.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3b0413e4f0 0x7f3b040aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 10 monclient(hunting): picked mon.noname-a con 0x7f3b0413e4f0 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:53.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3b041445e0 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:53.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b040b7870 con 0x7f3b041445e0 2023-07-29T14:05:53.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b040a91b0 con 0x7f3b040b74a0 2023-07-29T14:05:53.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:53.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b040a9490 con 0x7f3b0413e4f0 2023-07-29T14:05:53.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:53.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b0af2b640 10 monclient(hunting): authenticate will time out at 3675.767436s 2023-07-29T14:05:53.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b08cc7640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3b040b74a0 0x7f3b0413dde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.40:44978/0 (socket says 172.21.15.40:44978) 2023-07-29T14:05:53.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.635+0000 7f3b08cc7640 1 -- 172.21.15.40:0/3801230375 learned_addr learned my addr 172.21.15.40:0/3801230375 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:53.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.641+0000 7f3b03fff640 1 --2- 172.21.15.40:0/3801230375 >> v2:172.21.15.40:3300/0 conn(0x7f3b0413e4f0 0x7f3b040aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.641+0000 7f3b03fff640 10 monclient(hunting): get_auth_request con 0x7f3b0413e4f0 auth_method 0 2023-07-29T14:05:53.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.641+0000 7f3b03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:53.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.641+0000 7f3b03fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:53.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.641+0000 7f3b03fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:53.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.644+0000 7f3b03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:53.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.644+0000 7f3b03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.644+0000 7f3b03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.646+0000 7f3b03fff640 10 monclient(hunting): handle_auth_done global_id 14988 payload 274 2023-07-29T14:05:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.646+0000 7f3b03fff640 1 -- 172.21.15.40:0/3801230375 >> v1:172.21.15.45:6789/0 conn(0x7f3b040b74a0 legacy=0x7f3b0413dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.646+0000 7f3b03fff640 1 -- 172.21.15.40:0/3801230375 >> v1:172.21.15.40:6789/0 conn(0x7f3b041445e0 legacy=0x7f3b04144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.646+0000 7f3b03fff640 10 monclient: _finish_hunting 0 2023-07-29T14:05:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.646+0000 7f3b03fff640 1 monclient: found mon.noname-a 2023-07-29T14:05:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.646+0000 7f3b03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:53.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.646+0000 7f3b03fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.40:3300/0 2023-07-29T14:05:53.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.646+0000 7f3b03fff640 1 -- 172.21.15.40:0/3801230375 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b040a9770 con 0x7f3b0413e4f0 2023-07-29T14:05:53.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.648+0000 7f3b03fff640 1 --2- 172.21.15.40:0/3801230375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b0413e4f0 0x7f3b040aa150 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7f3af00097a0 tx=0x7f3af002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a931e5ab70f5e0d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 1 -- 172.21.15.40:0/3801230375 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af002e070 con 0x7f3b0413e4f0 2023-07-29T14:05:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-07-29T14:05:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 10 monclient: dump: 2023-07-29T14:05:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.729 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:53.729 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.729 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:53.729 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:53.729 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:53.730 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 10 monclient: _finish_auth 0 2023-07-29T14:05:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 10 monclient: _check_auth_tickets 2023-07-29T14:05:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 1 -- 172.21.15.40:0/3801230375 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af00315a0 con 0x7f3b0413e4f0 2023-07-29T14:05:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 1 -- 172.21.15.40:0/3801230375 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0031930 con 0x7f3b0413e4f0 2023-07-29T14:05:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b0af2b640 5 monclient: authenticate success, global_id 14988 2023-07-29T14:05:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b0af2b640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b0af2b640 4 set_mon_vals no callback set 2023-07-29T14:05:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-07-29T14:05:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.707+0000 7f3b037fe640 10 monclient: dump: 2023-07-29T14:05:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:53.736 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:53.736 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.708+0000 7f3b0af2b640 10 monclient: shutdown 2023-07-29T14:05:53.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.708+0000 7f3b0af2b640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:53.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.708+0000 7f3b0af2b640 1 -- 172.21.15.40:0/3801230375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b0413e4f0 msgr2=0x7f3b040aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.708+0000 7f3b0af2b640 1 --2- 172.21.15.40:0/3801230375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b0413e4f0 0x7f3b040aa150 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7f3af00097a0 tx=0x7f3af002b840 comp rx=0 tx=0).stop 2023-07-29T14:05:53.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.708+0000 7f3b0af2b640 1 -- 172.21.15.40:0/3801230375 shutdown_connections 2023-07-29T14:05:53.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.708+0000 7f3b0af2b640 1 --2- 172.21.15.40:0/3801230375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b0413e4f0 0x7f3b040aa150 unknown :-1 s=CLOSED pgs=3224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.708+0000 7f3b0af2b640 1 -- 172.21.15.40:0/3801230375 >> 172.21.15.40:0/3801230375 conn(0x7f3b04065b00 msgr2=0x7f3b04065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:53.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.708+0000 7f3b0af2b640 1 -- 172.21.15.40:0/3801230375 shutdown_connections 2023-07-29T14:05:53.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.708+0000 7f3b0af2b640 1 -- 172.21.15.40:0/3801230375 wait complete. 2023-07-29T14:05:53.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.709+0000 7f3b0af2b640 10 monclient: build_initial_monmap 2023-07-29T14:05:53.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.709+0000 7f3b0af2b640 10 monclient: monmap: 2023-07-29T14:05:53.738 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:53.738 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:53.738 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:53.738 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:53.738 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:53.738 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.738 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:53.739 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:53.739 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:53.739 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.710+0000 7f3b0af2b640 1 librados: starting msgr at 2023-07-29T14:05:53.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.710+0000 7f3b0af2b640 1 librados: starting objecter 2023-07-29T14:05:53.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.710+0000 7f3b0af2b640 1 Processor -- start 2023-07-29T14:05:53.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.710+0000 7f3b0af2b640 1 -- start start 2023-07-29T14:05:53.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.710+0000 7f3b0af2b640 1 librados: setting wanted keys 2023-07-29T14:05:53.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.710+0000 7f3b0af2b640 1 librados: calling monclient init 2023-07-29T14:05:53.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.710+0000 7f3b0af2b640 10 monclient: init 2023-07-29T14:05:53.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:53.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T14:05:53.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b040b74a0 0x7f3b040aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 10 monclient(hunting): picked mon.noname-a con 0x7f3b040b74a0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:53.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0413e4f0 0x7f3b041d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 10 monclient(hunting): picked mon.noname-b con 0x7f3b0413e4f0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3b041445e0 0x7f3b041d6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 10 monclient(hunting): picked mon.noname-c con 0x7f3b041445e0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040abda0 con 0x7f3b040b74a0 2023-07-29T14:05:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040ac080 con 0x7f3b0413e4f0 2023-07-29T14:05:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:53.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b041d59a0 con 0x7f3b041445e0 2023-07-29T14:05:53.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:53.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b0af2b640 10 monclient(hunting): authenticate will time out at 3675.843200s 2023-07-29T14:05:53.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.711+0000 7f3b094c8640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3b041445e0 0x7f3b041d6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.712+0000 7f3b094c8640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3b041445e0 0x7f3b041d6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56716/0 (socket says 172.21.15.40:56716) 2023-07-29T14:05:53.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.712+0000 7f3b094c8640 1 -- 172.21.15.40:0/1708397173 learned_addr learned my addr 172.21.15.40:0/1708397173 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:53.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.712+0000 7f3b094c8640 10 monclient(hunting): get_auth_request con 0x7f3b041445e0 auth_method 0 2023-07-29T14:05:53.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.712+0000 7f3b094c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:53.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.712+0000 7f3b094c8640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:53.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.712+0000 7f3b094c8640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:53.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.712+0000 7f3b094c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:53.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.712+0000 7f3b094c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:53.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.712+0000 7f3b094c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:53.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.712+0000 7f3b08cc7640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b040b74a0 0x7f3b040aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.712+0000 7f3b03fff640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0413e4f0 0x7f3b041d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.713+0000 7f3b03fff640 10 monclient(hunting): get_auth_request con 0x7f3b0413e4f0 auth_method 0 2023-07-29T14:05:53.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.713+0000 7f3b03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:53.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.713+0000 7f3b03fff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:53.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.713+0000 7f3b03fff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:53.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.713+0000 7f3b03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:53.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.713+0000 7f3b03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:53.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.713+0000 7f3b03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:53.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.713+0000 7f3b08cc7640 10 monclient(hunting): get_auth_request con 0x7f3b040b74a0 auth_method 0 2023-07-29T14:05:53.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.713+0000 7f3b08cc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:53.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.713+0000 7f3b08cc7640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:53.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.713+0000 7f3b08cc7640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:53.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.715+0000 7f3b094c8640 10 monclient(hunting): handle_auth_done global_id 16424 payload 824 2023-07-29T14:05:53.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.715+0000 7f3b094c8640 1 -- 172.21.15.40:0/1708397173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0413e4f0 msgr2=0x7f3b041d5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.715+0000 7f3b094c8640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0413e4f0 0x7f3b041d5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.715+0000 7f3b094c8640 1 -- 172.21.15.40:0/1708397173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b040b74a0 msgr2=0x7f3b040aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:53.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.715+0000 7f3b094c8640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b040b74a0 0x7f3b040aabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:53.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.715+0000 7f3b094c8640 10 monclient: _finish_hunting 0 2023-07-29T14:05:53.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.715+0000 7f3b094c8640 1 monclient: found mon.noname-c 2023-07-29T14:05:53.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.715+0000 7f3b094c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:53.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.715+0000 7f3b094c8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:05:53.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.715+0000 7f3b094c8640 1 -- 172.21.15.40:0/1708397173 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af0009400 con 0x7f3b041445e0 2023-07-29T14:05:53.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.718+0000 7f3b094c8640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3b041445e0 0x7f3b041d6400 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00b970 tx=0x7f3afc00be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:53.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.718+0000 7f3b017fa640 10 client.?.objecter ms_handle_connect 0x7f3b041445e0 2023-07-29T14:05:53.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.718+0000 7f3b017fa640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:53.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 1 -- 172.21.15.40:0/1708397173 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc00cb40 con 0x7f3b041445e0 2023-07-29T14:05:53.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:53.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:53.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 10 monclient: dump: 2023-07-29T14:05:53.756 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:53.756 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:53.757 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.757 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.757 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:53.757 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.757 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:53.757 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:53.758 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:53.758 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 10 monclient: _finish_auth 0 2023-07-29T14:05:53.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 10 monclient: _check_auth_tickets 2023-07-29T14:05:53.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:53.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 1 -- 172.21.15.40:0/1708397173 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc00cca0 con 0x7f3b041445e0 2023-07-29T14:05:53.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b0af2b640 5 monclient: authenticate success, global_id 16424 2023-07-29T14:05:53.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:53.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b0af2b640 10 monclient: _renew_subs 2023-07-29T14:05:53.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 1 -- 172.21.15.40:0/1708397173 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc014680 con 0x7f3b041445e0 2023-07-29T14:05:53.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b0af2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:53.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b0af2b640 1 -- 172.21.15.40:0/1708397173 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b04139ca0 con 0x7f3b041445e0 2023-07-29T14:05:53.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:53.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:53.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b027fc640 4 set_mon_vals no callback set 2023-07-29T14:05:53.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b017fa640 10 monclient: dump: 2023-07-29T14:05:53.760 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:53.760 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:53.761 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.761 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:53.761 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:53.761 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:53.761 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:53.761 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:53.761 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:53.761 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:53.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b0af2b640 10 client.16424.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:53.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b0af2b640 10 monclient: _renew_subs 2023-07-29T14:05:53.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b0af2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:53.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b0af2b640 1 -- 172.21.15.40:0/1708397173 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0413aaa0 con 0x7f3b041445e0 2023-07-29T14:05:53.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.721+0000 7f3b0af2b640 1 librados: init done 2023-07-29T14:05:53.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.722+0000 7f3b0af2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:53.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.722+0000 7f3b0af2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:53.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.722+0000 7f3b0af2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:05:53.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.722+0000 7f3b0af2b640 1 -- 172.21.15.40:0/1708397173 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0404e000 con 0x7f3b041445e0 2023-07-29T14:05:53.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.743+0000 7f3b017fa640 1 -- 172.21.15.40:0/1708397173 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f3afc003970 con 0x7f3b041445e0 2023-07-29T14:05:53.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.743+0000 7f3b017fa640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f3ae806bfb0 0x7f3ae806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:53.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.744+0000 7f3b08cc7640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f3ae806bfb0 0x7f3ae806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:53.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.744+0000 7f3b08cc7640 10 monclient: get_auth_request con 0x7f3ae806bfb0 auth_method 0 2023-07-29T14:05:53.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.744+0000 7f3b017fa640 10 client.16424.objecter ms_handle_connect 0x7f3ae806bfb0 2023-07-29T14:05:53.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.744+0000 7f3b08cc7640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f3ae806bfb0 0x7f3ae806e460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3af80058d0 tx=0x7f3af8009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:53.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.747+0000 7f3b017fa640 1 -- 172.21.15.40:0/1708397173 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f3afc05bad0 con 0x7f3b041445e0 2023-07-29T14:05:53.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.747+0000 7f3b017fa640 10 client.16424.objecter ms_dispatch 0x7f3b04000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:53.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.747+0000 7f3b017fa640 3 client.16424.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:53.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.747+0000 7f3b017fa640 3 client.16424.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:53.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:53.747+0000 7f3b017fa640 20 client.16424.objecter dump_active .. 0 homeless 2023-07-29T14:05:54.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.000+0000 7f3b017fa640 1 -- 172.21.15.40:0/1708397173 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f3afc053f00 con 0x7f3b041445e0 2023-07-29T14:05:54.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.000+0000 7f3b017fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:54.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.000+0000 7f3b017fa640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:54.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.186+0000 7f3b0af2b640 1 -- 172.21.15.40:0/1708397173 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b04145170 con 0x7f3ae806bfb0 2023-07-29T14:05:54.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.186+0000 7f3b017fa640 1 -- 172.21.15.40:0/1708397173 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3af8001dc0 con 0x7f3ae806bfb0 2023-07-29T14:05:54.187 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2023-07-29T14:05:54.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 10 librados: watch_flush enter 2023-07-29T14:05:54.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 10 librados: watch_flush exit 2023-07-29T14:05:54.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 20 client.16424.objecter shutdown clearing up homeless session... 2023-07-29T14:05:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 10 client.16424.objecter successfully canceled tick 2023-07-29T14:05:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 -- 172.21.15.40:0/1708397173 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f3ae806bfb0 msgr2=0x7f3ae806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f3ae806bfb0 0x7f3ae806e460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3af80058d0 tx=0x7f3af8009040 comp rx=0 tx=0).stop 2023-07-29T14:05:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 10 monclient: shutdown 2023-07-29T14:05:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 -- 172.21.15.40:0/1708397173 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3b041445e0 msgr2=0x7f3b041d6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3b041445e0 0x7f3b041d6400 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00b970 tx=0x7f3afc00be30 comp rx=0 tx=0).stop 2023-07-29T14:05:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 -- 172.21.15.40:0/1708397173 shutdown_connections 2023-07-29T14:05:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f3ae806bfb0 0x7f3ae806e460 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0413e4f0 0x7f3b041d5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3b041445e0 0x7f3b041d6400 unknown :-1 s=CLOSED pgs=3239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 --2- 172.21.15.40:0/1708397173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b040b74a0 0x7f3b040aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 -- 172.21.15.40:0/1708397173 >> 172.21.15.40:0/1708397173 conn(0x7f3b04065b00 msgr2=0x7f3b04068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 -- 172.21.15.40:0/1708397173 shutdown_connections 2023-07-29T14:05:54.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 -- 172.21.15.40:0/1708397173 wait complete. 2023-07-29T14:05:54.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.189+0000 7f3b0af2b640 1 librados: shutdown 2023-07-29T14:05:54.199 INFO:tasks.ceph:Scrubbing osd.7 2023-07-29T14:05:54.200 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-07-29T14:05:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.310+0000 7f6cd47a3640 10 monclient: get_monmap_and_config 2023-07-29T14:05:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.310+0000 7f6cd47a3640 10 monclient: build_initial_monmap 2023-07-29T14:05:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.310+0000 7f6cd47a3640 10 monclient: monmap: 2023-07-29T14:05:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:54.311043+0000 2023-07-29T14:05:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:54.311043+0000 2023-07-29T14:05:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:54.312 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.311+0000 7f6cd47a3640 1 Processor -- start 2023-07-29T14:05:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.311+0000 7f6cd47a3640 1 -- start start 2023-07-29T14:05:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.311+0000 7f6cd47a3640 10 monclient: init 2023-07-29T14:05:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2023-07-29T14:05:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6ccc0b3550 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6ccc0b3920 0x7f6ccc0af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 10 monclient(hunting): picked mon.noname-b con 0x7f6ccc0b3920 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6ccc0b0170 0x7f6ccc0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 10 monclient(hunting): picked mon.noname-c con 0x7f6ccc0b0170 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc0b0b30 con 0x7f6ccc0b3550 2023-07-29T14:05:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ccc13ec30 con 0x7f6ccc0b0170 2023-07-29T14:05:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc141e50 con 0x7f6ccc0b3920 2023-07-29T14:05:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd47a3640 10 monclient(hunting): authenticate will time out at 3676.444449s 2023-07-29T14:05:54.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.312+0000 7f6cd1d3e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6ccc0b3920 0x7f6ccc0af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.313+0000 7f6cd253f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6ccc0b3550 0x7f6ccc141740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38678/0 (socket says 172.21.15.40:38678) 2023-07-29T14:05:54.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.313+0000 7f6cd253f640 1 -- 172.21.15.40:0/3467400584 learned_addr learned my addr 172.21.15.40:0/3467400584 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:54.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.313+0000 7f6cd1d3e640 10 monclient(hunting): get_auth_request con 0x7f6ccc0b3920 auth_method 0 2023-07-29T14:05:54.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.313+0000 7f6cd1d3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.313+0000 7f6cd1d3e640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.313+0000 7f6cd1d3e640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.313+0000 7f6cd2d40640 1 --2- 172.21.15.40:0/3467400584 >> v2:172.21.15.45:3300/0 conn(0x7f6ccc0b0170 0x7f6ccc0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.313+0000 7f6cd1d3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.313+0000 7f6cd1d3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:54.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.313+0000 7f6cd1d3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:54.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.314+0000 7f6cd2d40640 10 monclient(hunting): get_auth_request con 0x7f6ccc0b0170 auth_method 0 2023-07-29T14:05:54.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.314+0000 7f6cd2d40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:54.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.314+0000 7f6cd2d40640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:54.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.314+0000 7f6cd2d40640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:54.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.316+0000 7f6cd1d3e640 10 monclient(hunting): handle_auth_done global_id 16427 payload 274 2023-07-29T14:05:54.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.316+0000 7f6cd1d3e640 1 -- 172.21.15.40:0/3467400584 >> v2:172.21.15.45:3300/0 conn(0x7f6ccc0b0170 msgr2=0x7f6ccc0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:54.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.316+0000 7f6cd1d3e640 1 --2- 172.21.15.40:0/3467400584 >> v2:172.21.15.45:3300/0 conn(0x7f6ccc0b0170 0x7f6ccc0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.316+0000 7f6cd1d3e640 1 -- 172.21.15.40:0/3467400584 >> v1:172.21.15.40:6789/0 conn(0x7f6ccc0b3550 legacy=0x7f6ccc141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.316+0000 7f6cd1d3e640 10 monclient: _finish_hunting 0 2023-07-29T14:05:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.316+0000 7f6cd1d3e640 1 monclient: found mon.noname-b 2023-07-29T14:05:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.316+0000 7f6cd1d3e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.316+0000 7f6cd1d3e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.316+0000 7f6cd1d3e640 1 -- 172.21.15.40:0/3467400584 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc0afd90 con 0x7f6ccc0b3920 2023-07-29T14:05:54.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.318+0000 7f6cd1d3e640 1 --2- 172.21.15.40:0/3467400584 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6ccc0b3920 0x7f6ccc0af850 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80097a0 tx=0x7f6cc802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b9ec09f5877033e server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:54.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 1 -- 172.21.15.40:0/3467400584 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc802e070 con 0x7f6ccc0b3920 2023-07-29T14:05:54.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:54.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:54.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 10 monclient: dump: 2023-07-29T14:05:54.326 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:54.326 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:54.326 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.326 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.326 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:54.327 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.327 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:54.327 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:54.327 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:54.327 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 10 monclient: _finish_auth 0 2023-07-29T14:05:54.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 10 monclient: _check_auth_tickets 2023-07-29T14:05:54.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:54.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 1 -- 172.21.15.40:0/3467400584 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc80315a0 con 0x7f6ccc0b3920 2023-07-29T14:05:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 1 -- 172.21.15.40:0/3467400584 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8031930 con 0x7f6ccc0b3920 2023-07-29T14:05:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd153d640 10 monclient: dump: 2023-07-29T14:05:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:54.332 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd47a3640 5 monclient: authenticate success, global_id 16427 2023-07-29T14:05:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd47a3640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:54.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.322+0000 7f6cd47a3640 4 set_mon_vals no callback set 2023-07-29T14:05:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 10 monclient: shutdown 2023-07-29T14:05:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3467400584 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6ccc0b3920 msgr2=0x7f6ccc0af850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 1 --2- 172.21.15.40:0/3467400584 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6ccc0b3920 0x7f6ccc0af850 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80097a0 tx=0x7f6cc802b840 comp rx=0 tx=0).stop 2023-07-29T14:05:54.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3467400584 shutdown_connections 2023-07-29T14:05:54.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 1 --2- 172.21.15.40:0/3467400584 >> v2:172.21.15.45:3300/0 conn(0x7f6ccc0b0170 0x7f6ccc0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 1 --2- 172.21.15.40:0/3467400584 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6ccc0b3920 0x7f6ccc0af850 unknown :-1 s=CLOSED pgs=3240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3467400584 >> 172.21.15.40:0/3467400584 conn(0x7f6ccc065920 msgr2=0x7f6ccc065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:54.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3467400584 shutdown_connections 2023-07-29T14:05:54.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3467400584 wait complete. 2023-07-29T14:05:54.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 10 monclient: build_initial_monmap 2023-07-29T14:05:54.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.323+0000 7f6cd47a3640 10 monclient: monmap: 2023-07-29T14:05:54.350 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:54.350 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:54.350 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:54.350 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:54.350 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:54.350 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.351 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:54.351 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:54.351 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:54.352 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.324+0000 7f6cd47a3640 1 librados: starting msgr at 2023-07-29T14:05:54.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.324+0000 7f6cd47a3640 1 librados: starting objecter 2023-07-29T14:05:54.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.324+0000 7f6cd47a3640 1 Processor -- start 2023-07-29T14:05:54.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.324+0000 7f6cd47a3640 1 -- start start 2023-07-29T14:05:54.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.324+0000 7f6cd47a3640 1 librados: setting wanted keys 2023-07-29T14:05:54.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.324+0000 7f6cd47a3640 1 librados: calling monclient init 2023-07-29T14:05:54.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.324+0000 7f6cd47a3640 10 monclient: init 2023-07-29T14:05:54.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.324+0000 7f6cd47a3640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:54.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.324+0000 7f6cd47a3640 10 monclient: _add_conns ranks=[0,1,2] 2023-07-29T14:05:54.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.324+0000 7f6cd47a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ccc0b0170 0x7f6ccc13cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 10 monclient(hunting): picked mon.noname-a con 0x7f6ccc0b0170 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:54.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ccc0b3550 0x7f6ccc1d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 10 monclient(hunting): picked mon.noname-b con 0x7f6ccc0b3550 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6ccc0b3920 0x7f6ccc0ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 10 monclient(hunting): picked mon.noname-c con 0x7f6ccc0b3920 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc1d9ab0 con 0x7f6ccc0b0170 2023-07-29T14:05:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0ab290 con 0x7f6ccc0b3550 2023-07-29T14:05:54.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:54.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc144f40 con 0x7f6ccc0b3920 2023-07-29T14:05:54.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:54.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd47a3640 10 monclient(hunting): authenticate will time out at 3676.457152s 2023-07-29T14:05:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd2d40640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6ccc0b3920 0x7f6ccc0ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd253f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ccc0b0170 0x7f6ccc13cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd1d3e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ccc0b3550 0x7f6ccc1d8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd2d40640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6ccc0b3920 0x7f6ccc0ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56742/0 (socket says 172.21.15.40:56742) 2023-07-29T14:05:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd2d40640 1 -- 172.21.15.40:0/3308672063 learned_addr learned my addr 172.21.15.40:0/3308672063 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd2d40640 10 monclient(hunting): get_auth_request con 0x7f6ccc0b3920 auth_method 0 2023-07-29T14:05:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd2d40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd2d40640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.325+0000 7f6cd2d40640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd1d3e640 10 monclient(hunting): get_auth_request con 0x7f6ccc0b3550 auth_method 0 2023-07-29T14:05:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd1d3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd1d3e640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd1d3e640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd253f640 10 monclient(hunting): get_auth_request con 0x7f6ccc0b0170 auth_method 0 2023-07-29T14:05:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd253f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd253f640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd253f640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd2d40640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd2d40640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd2d40640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd1d3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd1d3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd1d3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd253f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd253f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.326+0000 7f6cd253f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.329+0000 7f6cd1d3e640 10 monclient(hunting): handle_auth_done global_id 20416 payload 824 2023-07-29T14:05:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.329+0000 7f6cd1d3e640 1 -- 172.21.15.40:0/3308672063 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6ccc0b3920 msgr2=0x7f6ccc0ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.329+0000 7f6cd1d3e640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6ccc0b3920 0x7f6ccc0ac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.329+0000 7f6cd1d3e640 1 -- 172.21.15.40:0/3308672063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ccc0b0170 msgr2=0x7f6ccc13cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.329+0000 7f6cd1d3e640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ccc0b0170 0x7f6ccc13cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.329+0000 7f6cd1d3e640 10 monclient: _finish_hunting 0 2023-07-29T14:05:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.329+0000 7f6cd1d3e640 1 monclient: found mon.noname-b 2023-07-29T14:05:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.329+0000 7f6cd1d3e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.329+0000 7f6cd1d3e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T14:05:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.329+0000 7f6cd1d3e640 1 -- 172.21.15.40:0/3308672063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc8009400 con 0x7f6ccc0b3550 2023-07-29T14:05:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.331+0000 7f6cd1d3e640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ccc0b3550 0x7f6ccc1d8fb0 secure :-1 s=READY pgs=4790 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8033000 tx=0x7f6cc80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.331+0000 7f6cc2ffd640 10 client.?.objecter ms_handle_connect 0x7f6ccc0b3550 2023-07-29T14:05:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.331+0000 7f6cc2ffd640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 1 -- 172.21.15.40:0/3308672063 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8032900 con 0x7f6ccc0b3550 2023-07-29T14:05:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 10 monclient: dump: 2023-07-29T14:05:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.371 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:54.371 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.371 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:54.372 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 10 monclient: _finish_auth 0 2023-07-29T14:05:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 1 -- 172.21.15.40:0/3308672063 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc8032aa0 con 0x7f6ccc0b3550 2023-07-29T14:05:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cd47a3640 5 monclient: authenticate success, global_id 20416 2023-07-29T14:05:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cd47a3640 10 monclient: _renew_subs 2023-07-29T14:05:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cd47a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T14:05:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3308672063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc1d94f0 con 0x7f6ccc0b3550 2023-07-29T14:05:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 1 -- 172.21.15.40:0/3308672063 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8032e50 con 0x7f6ccc0b3550 2023-07-29T14:05:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T14:05:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cc2ffd640 10 monclient: dump: 2023-07-29T14:05:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:54.375 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cd0d3c640 4 set_mon_vals no callback set 2023-07-29T14:05:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cd47a3640 10 client.20416.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cd47a3640 10 monclient: _renew_subs 2023-07-29T14:05:54.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cd47a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T14:05:54.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3308672063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc0acf30 con 0x7f6ccc0b3550 2023-07-29T14:05:54.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.335+0000 7f6cd47a3640 1 librados: init done 2023-07-29T14:05:54.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.336+0000 7f6cd47a3640 10 client.20416.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:54.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.336+0000 7f6cd47a3640 15 client.20416.objecter _session_command_op_assign -1 1 2023-07-29T14:05:54.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.336+0000 7f6cd47a3640 20 client.20416.objecter _get_session osd=-1 returning homeless 2023-07-29T14:05:54.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.336+0000 7f6cd47a3640 10 client.20416.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:54.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.336+0000 7f6cd47a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T14:05:54.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.336+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3308672063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6ccc140f40 con 0x7f6ccc0b3550 2023-07-29T14:05:54.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.354+0000 7f6cc2ffd640 1 -- 172.21.15.40:0/3308672063 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f6cc804e430 con 0x7f6ccc0b3550 2023-07-29T14:05:54.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.355+0000 7f6cc2ffd640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f6cb006bfb0 0x7f6cb006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.355+0000 7f6cd253f640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f6cb006bfb0 0x7f6cb006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.355+0000 7f6cd253f640 10 monclient: get_auth_request con 0x7f6cb006bfb0 auth_method 0 2023-07-29T14:05:54.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.356+0000 7f6cd253f640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f6cb006bfb0 0x7f6cb006e460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc005880 tx=0x7f6cbc009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:54.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.356+0000 7f6cc2ffd640 10 client.20416.objecter ms_handle_connect 0x7f6cb006bfb0 2023-07-29T14:05:54.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 1 -- 172.21.15.40:0/3308672063 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f6cc807d8a0 con 0x7f6ccc0b3550 2023-07-29T14:05:54.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 10 client.20416.objecter ms_dispatch 0x7f6ccc000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:54.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 3 client.20416.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:54.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 3 client.20416.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:54.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 10 client.20416.objecter checking command 1 2023-07-29T14:05:54.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] conn(0x7f6cb00737f0 0x7f6cb0075c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 20 client.20416.objecter _get_session s=0x7f6cb0073120 osd=7 3 2023-07-29T14:05:54.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 20 client.20416.objecter put_session s=0x7f6cb0073120 osd=7 3 2023-07-29T14:05:54.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 15 client.20416.objecter _session_command_op_remove -1 1 2023-07-29T14:05:54.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 20 client.20416.objecter _get_session s=0x7f6cb0073120 osd=7 3 2023-07-29T14:05:54.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 20 client.20416.objecter get_session s=0x7f6cb0073120 osd=7 3 2023-07-29T14:05:54.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 15 client.20416.objecter _session_command_op_assign 7 1 2023-07-29T14:05:54.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 20 client.20416.objecter put_session s=0x7f6cb0073120 osd=7 4 2023-07-29T14:05:54.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 10 client.20416.objecter _send_command 1 2023-07-29T14:05:54.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 1 -- 172.21.15.40:0/3308672063 --> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cb0076350 con 0x7f6cb00737f0 2023-07-29T14:05:54.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 20 client.20416.objecter dump_active .. 0 homeless 2023-07-29T14:05:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 1 -- 172.21.15.40:0/3308672063 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=1004) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cc8075cd0 con 0x7f6ccc0b3550 2023-07-29T14:05:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cc2ffd640 10 monclient: handle_get_version_reply finishing 1 version 1004 2023-07-29T14:05:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.358+0000 7f6cd2d40640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] conn(0x7f6cb00737f0 0x7f6cb0075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.359+0000 7f6cd2d40640 10 monclient: get_auth_request con 0x7f6cb00737f0 auth_method 0 2023-07-29T14:05:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.361+0000 7f6cd2d40640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] conn(0x7f6cb00737f0 0x7f6cb0075c80 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.361+0000 7f6cc2ffd640 10 client.20416.objecter ms_handle_connect 0x7f6cb00737f0 2023-07-29T14:05:54.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.385+0000 7f6cc2ffd640 1 -- 172.21.15.40:0/3308672063 <== osd.7 v2:172.21.15.45:6802/85182 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6cb0076350 con 0x7f6cb00737f0 2023-07-29T14:05:54.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.385+0000 7f6cc2ffd640 10 client.20416.objecter ms_dispatch 0x7f6ccc000b90 command_reply(tid 1: 0 ) v1 2023-07-29T14:05:54.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.385+0000 7f6cc2ffd640 10 client.20416.objecter _finish_command 1 = system:0 2023-07-29T14:05:54.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.385+0000 7f6cc2ffd640 20 client.20416.objecter put_session s=0x7f6cb0073120 osd=7 4 2023-07-29T14:05:54.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.385+0000 7f6cc2ffd640 15 client.20416.objecter _session_command_op_remove 7 1 2023-07-29T14:05:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.398+0000 7f6cd47a3640 10 client.20416.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-07-29T14:05:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.398+0000 7f6cd47a3640 15 client.20416.objecter _session_command_op_assign -1 2 2023-07-29T14:05:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.398+0000 7f6cd47a3640 20 client.20416.objecter _get_session s=0x7f6cb0073120 osd=7 3 2023-07-29T14:05:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.398+0000 7f6cd47a3640 20 client.20416.objecter put_session s=0x7f6cb0073120 osd=7 3 2023-07-29T14:05:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.398+0000 7f6cd47a3640 20 client.20416.objecter _get_session s=0x7f6cb0073120 osd=7 3 2023-07-29T14:05:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.398+0000 7f6cd47a3640 15 client.20416.objecter _session_command_op_remove -1 2 2023-07-29T14:05:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.398+0000 7f6cd47a3640 20 client.20416.objecter get_session s=0x7f6cb0073120 osd=7 3 2023-07-29T14:05:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.398+0000 7f6cd47a3640 15 client.20416.objecter _session_command_op_assign 7 2 2023-07-29T14:05:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.398+0000 7f6cd47a3640 20 client.20416.objecter put_session s=0x7f6cb0073120 osd=7 4 2023-07-29T14:05:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.398+0000 7f6cd47a3640 10 client.20416.objecter _send_command 2 2023-07-29T14:05:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.398+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3308672063 --> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6ccc0af150 con 0x7f6cb00737f0 2023-07-29T14:05:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.417+0000 7f6cc2ffd640 1 -- 172.21.15.40:0/3308672063 <== osd.7 v2:172.21.15.45:6802/85182 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6cc4007530 con 0x7f6cb00737f0 2023-07-29T14:05:54.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.417+0000 7f6cc2ffd640 10 client.20416.objecter ms_dispatch 0x7f6ccc000b90 command_reply(tid 2: 0 ) v1 2023-07-29T14:05:54.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.417+0000 7f6cc2ffd640 10 client.20416.objecter _finish_command 2 = system:0 2023-07-29T14:05:54.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.417+0000 7f6cc2ffd640 20 client.20416.objecter put_session s=0x7f6cb0073120 osd=7 4 2023-07-29T14:05:54.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.417+0000 7f6cc2ffd640 15 client.20416.objecter _session_command_op_remove 7 2 2023-07-29T14:05:54.418 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-29T14:05:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-29T14:05:54.419 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-29T14:05:54.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.417+0000 7f6cd47a3640 10 librados: watch_flush enter 2023-07-29T14:05:54.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 10 librados: watch_flush exit 2023-07-29T14:05:54.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 10 client.20416.objecter close_session for osd.7 2023-07-29T14:05:54.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] conn(0x7f6cb00737f0 msgr2=0x7f6cb0075c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:6802/85182,v1:172.21.15.45:6803/85182] conn(0x7f6cb00737f0 0x7f6cb0075c80 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 20 client.20416.objecter put_session s=0x7f6cb0073120 osd=7 1 2023-07-29T14:05:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 20 client.20416.objecter shutdown clearing up homeless session... 2023-07-29T14:05:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 10 client.20416.objecter successfully canceled tick 2023-07-29T14:05:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f6cb006bfb0 msgr2=0x7f6cb006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f6cb006bfb0 0x7f6cb006e460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc005880 tx=0x7f6cbc009040 comp rx=0 tx=0).stop 2023-07-29T14:05:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 10 monclient: shutdown 2023-07-29T14:05:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ccc0b3550 msgr2=0x7f6ccc1d8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 1 --2- 172.21.15.40:0/3308672063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ccc0b3550 0x7f6ccc1d8fb0 secure :-1 s=READY pgs=4790 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8033000 tx=0x7f6cc80046c0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd2d40640 1 -- 172.21.15.40:0/3308672063 reap_dead start 2023-07-29T14:05:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3308672063 shutdown_connections 2023-07-29T14:05:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3308672063 >> 172.21.15.40:0/3308672063 conn(0x7f6ccc065920 msgr2=0x7f6ccc0b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3308672063 shutdown_connections 2023-07-29T14:05:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 1 -- 172.21.15.40:0/3308672063 wait complete. 2023-07-29T14:05:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.418+0000 7f6cd47a3640 1 librados: shutdown 2023-07-29T14:05:54.428 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-07-29T14:05:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.535+0000 7fe96ff24640 10 monclient: get_monmap_and_config 2023-07-29T14:05:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.535+0000 7fe96ff24640 10 monclient: build_initial_monmap 2023-07-29T14:05:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.535+0000 7fe96ff24640 10 monclient: monmap: 2023-07-29T14:05:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:54.535916+0000 2023-07-29T14:05:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:54.535916+0000 2023-07-29T14:05:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:54.538 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.536+0000 7fe96ff24640 1 Processor -- start 2023-07-29T14:05:54.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.536+0000 7fe96ff24640 1 -- start start 2023-07-29T14:05:54.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.536+0000 7fe96ff24640 10 monclient: init 2023-07-29T14:05:54.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2023-07-29T14:05:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe9680afea0 0x7fe96813bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 10 monclient(hunting): picked mon.noname-a con 0x7fe9680afea0 addr v2:172.21.15.40:3300/0 2023-07-29T14:05:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe96813c0c0 0x7fe968141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 10 monclient(hunting): picked mon.noname-c con 0x7fe96813c0c0 addr v2:172.21.15.45:3300/0 2023-07-29T14:05:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe9680a8c30 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:54.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9680b0270 con 0x7fe9680a8c30 2023-07-29T14:05:54.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:54.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe968142ad0 con 0x7fe9680afea0 2023-07-29T14:05:54.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:54.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9680af010 con 0x7fe96813c0c0 2023-07-29T14:05:54.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:54.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96ff24640 10 monclient(hunting): authenticate will time out at 3676.669298s 2023-07-29T14:05:54.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96dcc0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe9680afea0 0x7fe96813bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.537+0000 7fe96d4bf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe96813c0c0 0x7fe968141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96d4bf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe96813c0c0 0x7fe968141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.40:35498/0 (socket says 172.21.15.40:35498) 2023-07-29T14:05:54.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96d4bf640 1 -- 172.21.15.40:0/128159369 learned_addr learned my addr 172.21.15.40:0/128159369 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:54.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96d4bf640 10 monclient(hunting): get_auth_request con 0x7fe96813c0c0 auth_method 0 2023-07-29T14:05:54.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96d4bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:54.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96d4bf640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:54.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96d4bf640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:54.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96dcc0640 10 monclient(hunting): get_auth_request con 0x7fe9680afea0 auth_method 0 2023-07-29T14:05:54.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96dcc0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:54.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96dcc0640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:54.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96dcc0640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:54.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96dcc0640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:54.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96dcc0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:54.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96dcc0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:54.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96d4bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:54.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96d4bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:54.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.538+0000 7fe96d4bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:54.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.540+0000 7fe96d4bf640 10 monclient(hunting): handle_auth_done global_id 20419 payload 274 2023-07-29T14:05:54.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.540+0000 7fe96d4bf640 1 -- 172.21.15.40:0/128159369 >> v2:172.21.15.40:3300/0 conn(0x7fe9680afea0 msgr2=0x7fe96813bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:54.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.540+0000 7fe96d4bf640 1 --2- 172.21.15.40:0/128159369 >> v2:172.21.15.40:3300/0 conn(0x7fe9680afea0 0x7fe96813bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.540+0000 7fe96d4bf640 1 -- 172.21.15.40:0/128159369 >> v1:172.21.15.45:6789/0 conn(0x7fe9680a8c30 legacy=0x7fe9681423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:05:54.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.540+0000 7fe96d4bf640 10 monclient: _finish_hunting 0 2023-07-29T14:05:54.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.540+0000 7fe96d4bf640 1 monclient: found mon.noname-c 2023-07-29T14:05:54.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.540+0000 7fe96d4bf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:54.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.540+0000 7fe96d4bf640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.45:3300/0 2023-07-29T14:05:54.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.540+0000 7fe96d4bf640 1 -- 172.21.15.40:0/128159369 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9680af2b0 con 0x7fe96813c0c0 2023-07-29T14:05:54.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.542+0000 7fe96d4bf640 1 --2- 172.21.15.40:0/128159369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe96813c0c0 0x7fe968141e80 secure :-1 s=READY pgs=4791 cs=0 l=1 rev1=1 crypto rx=0x7fe9680ae200 tx=0x7fe95802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86c2aa7a91844caf server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:54.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 1 -- 172.21.15.40:0/128159369 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958030860 con 0x7fe96813c0c0 2023-07-29T14:05:54.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:54.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:05:54.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 10 monclient: dump: 2023-07-29T14:05:54.549 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:54.549 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:54.549 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.549 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.549 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:54.549 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.549 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:54.550 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:54.550 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:54.550 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 10 monclient: _finish_auth 0 2023-07-29T14:05:54.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 10 monclient: _check_auth_tickets 2023-07-29T14:05:54.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:54.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 1 -- 172.21.15.40:0/128159369 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9580309c0 con 0x7fe96813c0c0 2023-07-29T14:05:54.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:54.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 1 -- 172.21.15.40:0/128159369 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958030d50 con 0x7fe96813c0c0 2023-07-29T14:05:54.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ff24640 5 monclient: authenticate success, global_id 20419 2023-07-29T14:05:54.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ff24640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:54.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ff24640 4 set_mon_vals no callback set 2023-07-29T14:05:54.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:54.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T14:05:54.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.546+0000 7fe96ccbe640 10 monclient: dump: 2023-07-29T14:05:54.552 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:54.552 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:54.552 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.553 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.553 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:54.553 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.553 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:54.553 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:54.553 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:54.554 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 10 monclient: shutdown 2023-07-29T14:05:54.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:54.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 1 -- 172.21.15.40:0/128159369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe96813c0c0 msgr2=0x7fe968141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 1 --2- 172.21.15.40:0/128159369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe96813c0c0 0x7fe968141e80 secure :-1 s=READY pgs=4791 cs=0 l=1 rev1=1 crypto rx=0x7fe9680ae200 tx=0x7fe95802b9a0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 1 -- 172.21.15.40:0/128159369 shutdown_connections 2023-07-29T14:05:54.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 1 --2- 172.21.15.40:0/128159369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe96813c0c0 0x7fe968141e80 unknown :-1 s=CLOSED pgs=4791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 1 --2- 172.21.15.40:0/128159369 >> v2:172.21.15.40:3300/0 conn(0x7fe9680afea0 0x7fe96813bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 1 -- 172.21.15.40:0/128159369 >> 172.21.15.40:0/128159369 conn(0x7fe968065b00 msgr2=0x7fe968065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 1 -- 172.21.15.40:0/128159369 shutdown_connections 2023-07-29T14:05:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 1 -- 172.21.15.40:0/128159369 wait complete. 2023-07-29T14:05:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 10 monclient: build_initial_monmap 2023-07-29T14:05:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.547+0000 7fe96ff24640 10 monclient: monmap: 2023-07-29T14:05:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:54.557 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 librados: starting msgr at 2023-07-29T14:05:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 librados: starting objecter 2023-07-29T14:05:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 Processor -- start 2023-07-29T14:05:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 -- start start 2023-07-29T14:05:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 librados: setting wanted keys 2023-07-29T14:05:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 librados: calling monclient init 2023-07-29T14:05:54.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 10 monclient: init 2023-07-29T14:05:54.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:54.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T14:05:54.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9680a8c30 0x7fe9680b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 10 monclient(hunting): picked mon.noname-c con 0x7fe9680a8c30 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9680afea0 0x7fe9680b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 10 monclient(hunting): picked mon.noname-b con 0x7fe9680afea0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe96813c0c0 0x7fe9681d2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 10 monclient(hunting): picked mon.noname-a con 0x7fe96813c0c0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9681cf780 con 0x7fe96813c0c0 2023-07-29T14:05:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9681d1ba0 con 0x7fe9680afea0 2023-07-29T14:05:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:54.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680b25c0 con 0x7fe9680a8c30 2023-07-29T14:05:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.548+0000 7fe96ff24640 10 monclient(hunting): authenticate will time out at 3676.680905s 2023-07-29T14:05:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96e4c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe96813c0c0 0x7fe9681d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96dcc0640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9680a8c30 0x7fe9680b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96dcc0640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9680a8c30 0x7fe9680b14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56758/0 (socket says 172.21.15.40:56758) 2023-07-29T14:05:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96dcc0640 1 -- 172.21.15.40:0/1657696789 learned_addr learned my addr 172.21.15.40:0/1657696789 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96dcc0640 10 monclient(hunting): get_auth_request con 0x7fe9680a8c30 auth_method 0 2023-07-29T14:05:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96dcc0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96dcc0640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96dcc0640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96e4c1640 10 monclient(hunting): get_auth_request con 0x7fe96813c0c0 auth_method 0 2023-07-29T14:05:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96d4bf640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9680afea0 0x7fe9680b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96e4c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96e4c1640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96e4c1640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96d4bf640 10 monclient(hunting): get_auth_request con 0x7fe9680afea0 auth_method 0 2023-07-29T14:05:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96d4bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96d4bf640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.549+0000 7fe96d4bf640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.550+0000 7fe96dcc0640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.550+0000 7fe96dcc0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.550+0000 7fe96dcc0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.550+0000 7fe96e4c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.550+0000 7fe96e4c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.550+0000 7fe96e4c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.550+0000 7fe96d4bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.550+0000 7fe96d4bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.550+0000 7fe96d4bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.553+0000 7fe96d4bf640 10 monclient(hunting): handle_auth_done global_id 20422 payload 824 2023-07-29T14:05:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.553+0000 7fe96d4bf640 1 -- 172.21.15.40:0/1657696789 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9680a8c30 msgr2=0x7fe9680b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.553+0000 7fe96d4bf640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9680a8c30 0x7fe9680b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.553+0000 7fe96d4bf640 1 -- 172.21.15.40:0/1657696789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe96813c0c0 msgr2=0x7fe9681d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.553+0000 7fe96d4bf640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe96813c0c0 0x7fe9681d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.553+0000 7fe96d4bf640 10 monclient: _finish_hunting 0 2023-07-29T14:05:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.553+0000 7fe96d4bf640 1 monclient: found mon.noname-b 2023-07-29T14:05:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.553+0000 7fe96d4bf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.553+0000 7fe96d4bf640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T14:05:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.553+0000 7fe96d4bf640 1 -- 172.21.15.40:0/1657696789 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580094d0 con 0x7fe9680afea0 2023-07-29T14:05:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.553+0000 7fe96e4c1640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe96813c0c0 0x7fe9681d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-29T14:05:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.555+0000 7fe96d4bf640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9680afea0 0x7fe9680b2080 secure :-1 s=READY pgs=4792 cs=0 l=1 rev1=1 crypto rx=0x7fe958033000 tx=0x7fe9580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.555+0000 7fe95e7fc640 10 client.?.objecter ms_handle_connect 0x7fe9680afea0 2023-07-29T14:05:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.555+0000 7fe95e7fc640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.557+0000 7fe95e7fc640 1 -- 172.21.15.40:0/1657696789 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95803e070 con 0x7fe9680afea0 2023-07-29T14:05:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.557+0000 7fe95e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.557+0000 7fe95e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.557+0000 7fe95e7fc640 10 monclient: dump: 2023-07-29T14:05:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:54.575 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:54.575 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.557+0000 7fe95e7fc640 10 monclient: _finish_auth 0 2023-07-29T14:05:54.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.557+0000 7fe95e7fc640 10 monclient: _check_auth_tickets 2023-07-29T14:05:54.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.557+0000 7fe95e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:54.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.557+0000 7fe96ff24640 5 monclient: authenticate success, global_id 20422 2023-07-29T14:05:54.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe96ff24640 10 monclient: _renew_subs 2023-07-29T14:05:54.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe96ff24640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T14:05:54.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe96ff24640 1 -- 172.21.15.40:0/1657696789 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9680aca80 con 0x7fe9680afea0 2023-07-29T14:05:54.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe96ff24640 10 client.20422.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:54.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe96ff24640 10 monclient: _renew_subs 2023-07-29T14:05:54.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe96ff24640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T14:05:54.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe96ff24640 1 -- 172.21.15.40:0/1657696789 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9681d2590 con 0x7fe9680afea0 2023-07-29T14:05:54.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe96ff24640 1 librados: init done 2023-07-29T14:05:54.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe95e7fc640 1 -- 172.21.15.40:0/1657696789 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe958039780 con 0x7fe9680afea0 2023-07-29T14:05:54.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe95e7fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:54.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe95e7fc640 1 -- 172.21.15.40:0/1657696789 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958039af0 con 0x7fe9680afea0 2023-07-29T14:05:54.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe95e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe95e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T14:05:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe95e7fc640 10 monclient: dump: 2023-07-29T14:05:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:54.583 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:54.584 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.584 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:54.584 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:54.584 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:54.584 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:54.584 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:54.585 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:54.585 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:54.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe95ffff640 4 set_mon_vals no callback set 2023-07-29T14:05:54.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.558+0000 7fe96ff24640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:54.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.559+0000 7fe96ff24640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:54.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.559+0000 7fe96ff24640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T14:05:54.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.559+0000 7fe96ff24640 1 -- 172.21.15.40:0/1657696789 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe950005040 con 0x7fe9680afea0 2023-07-29T14:05:54.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.578+0000 7fe95e7fc640 1 -- 172.21.15.40:0/1657696789 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7fe958042070 con 0x7fe9680afea0 2023-07-29T14:05:54.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.578+0000 7fe95e7fc640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe94c06bf60 0x7fe94c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:54.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.579+0000 7fe96dcc0640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe94c06bf60 0x7fe94c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:54.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.579+0000 7fe96dcc0640 10 monclient: get_auth_request con 0x7fe94c06bf60 auth_method 0 2023-07-29T14:05:54.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.579+0000 7fe96dcc0640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe94c06bf60 0x7fe94c06e410 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe964009c00 tx=0x7fe964009190 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:54.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.579+0000 7fe95e7fc640 10 client.20422.objecter ms_handle_connect 0x7fe94c06bf60 2023-07-29T14:05:54.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.581+0000 7fe95e7fc640 1 -- 172.21.15.40:0/1657696789 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7fe95807e250 con 0x7fe9680afea0 2023-07-29T14:05:54.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.581+0000 7fe95e7fc640 10 client.20422.objecter ms_dispatch 0x7fe968000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:54.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.581+0000 7fe95e7fc640 3 client.20422.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:54.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.581+0000 7fe95e7fc640 3 client.20422.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:54.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.581+0000 7fe95e7fc640 20 client.20422.objecter dump_active .. 0 homeless 2023-07-29T14:05:54.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.831+0000 7fe95e7fc640 1 -- 172.21.15.40:0/1657696789 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7fe958076680 con 0x7fe9680afea0 2023-07-29T14:05:54.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.831+0000 7fe95e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:54.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:54.831+0000 7fe95e7fc640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:55.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.040+0000 7fe96ff24640 1 -- 172.21.15.40:0/1657696789 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fe950000bc0 con 0x7fe94c06bf60 2023-07-29T14:05:55.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.041+0000 7fe95e7fc640 1 -- 172.21.15.40:0/1657696789 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe950000bc0 con 0x7fe94c06bf60 2023-07-29T14:05:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2023-07-29T14:05:55.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 10 librados: watch_flush enter 2023-07-29T14:05:55.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 10 librados: watch_flush exit 2023-07-29T14:05:55.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 20 client.20422.objecter shutdown clearing up homeless session... 2023-07-29T14:05:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 10 client.20422.objecter successfully canceled tick 2023-07-29T14:05:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 -- 172.21.15.40:0/1657696789 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe94c06bf60 msgr2=0x7fe94c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe94c06bf60 0x7fe94c06e410 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe964009c00 tx=0x7fe964009190 comp rx=0 tx=0).stop 2023-07-29T14:05:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 10 monclient: shutdown 2023-07-29T14:05:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 -- 172.21.15.40:0/1657696789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9680afea0 msgr2=0x7fe9680b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9680afea0 0x7fe9680b2080 secure :-1 s=READY pgs=4792 cs=0 l=1 rev1=1 crypto rx=0x7fe958033000 tx=0x7fe9580046c0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 -- 172.21.15.40:0/1657696789 shutdown_connections 2023-07-29T14:05:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7fe94c06bf60 0x7fe94c06e410 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9680afea0 0x7fe9680b2080 unknown :-1 s=CLOSED pgs=4792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe96813c0c0 0x7fe9681d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 --2- 172.21.15.40:0/1657696789 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9680a8c30 0x7fe9680b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 -- 172.21.15.40:0/1657696789 >> 172.21.15.40:0/1657696789 conn(0x7fe968065b00 msgr2=0x7fe968137b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 -- 172.21.15.40:0/1657696789 shutdown_connections 2023-07-29T14:05:55.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 -- 172.21.15.40:0/1657696789 wait complete. 2023-07-29T14:05:55.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.043+0000 7fe96ff24640 1 librados: shutdown 2023-07-29T14:05:55.054 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-29T14:05:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.164+0000 7f4474275640 10 monclient: get_monmap_and_config 2023-07-29T14:05:55.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.164+0000 7f4474275640 10 monclient: build_initial_monmap 2023-07-29T14:05:55.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.165+0000 7f4474275640 10 monclient: monmap: 2023-07-29T14:05:55.165 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:55.165 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:55.165 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:05:55.165197+0000 2023-07-29T14:05:55.166 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:05:55.165197+0000 2023-07-29T14:05:55.166 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:55.166 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:55.166 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:05:55.166 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:05:55.166 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:05:55.166 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:05:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:05:55.167 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.165+0000 7f4474275640 1 Processor -- start 2023-07-29T14:05:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.165+0000 7f4474275640 1 -- start start 2023-07-29T14:05:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.165+0000 7f4474275640 10 monclient: init 2023-07-29T14:05:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:55.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2023-07-29T14:05:55.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f446c0b74a0 addr v1:172.21.15.40:6789/0 2023-07-29T14:05:55.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c13e4f0 0x7f446c0aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:55.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 10 monclient(hunting): picked mon.noname-b con 0x7f446c13e4f0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f446c1445e0 addr v1:172.21.15.45:6789/0 2023-07-29T14:05:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c0b7870 con 0x7f446c0b74a0 2023-07-29T14:05:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c0a91b0 con 0x7f446c1445e0 2023-07-29T14:05:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0a9490 con 0x7f446c13e4f0 2023-07-29T14:05:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:55.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.166+0000 7f4474275640 10 monclient(hunting): authenticate will time out at 3677.298581s 2023-07-29T14:05:55.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.167+0000 7f4472011640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f446c0b74a0 0x7f446c13dde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38688/0 (socket says 172.21.15.40:38688) 2023-07-29T14:05:55.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.167+0000 7f4472011640 1 -- 172.21.15.40:0/2497766228 learned_addr learned my addr 172.21.15.40:0/2497766228 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-07-29T14:05:55.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.167+0000 7f4471810640 1 --2- 172.21.15.40:0/2497766228 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c13e4f0 0x7f446c0aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.167+0000 7f4471810640 10 monclient(hunting): get_auth_request con 0x7f446c13e4f0 auth_method 0 2023-07-29T14:05:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.167+0000 7f4471810640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.167+0000 7f4471810640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.167+0000 7f4471810640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.168+0000 7f4471810640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.168+0000 7f4471810640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.168+0000 7f4471810640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 1 -- 172.21.15.40:0/2497766228 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308883477 0 0) 0x7f446c0b7870 con 0x7f446c0b74a0 2023-07-29T14:05:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 10 monclient(hunting): _negotiate 2023-07-29T14:05:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 10 monclient(hunting): my global_id is 15006 2023-07-29T14:05:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 1 -- 172.21.15.40:0/2497766228 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4458003520 con 0x7f446c0b74a0 2023-07-29T14:05:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 1 -- 172.21.15.40:0/2497766228 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678759409 0 0) 0x7f446c0a91b0 con 0x7f446c1445e0 2023-07-29T14:05:55.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 10 monclient(hunting): _negotiate 2023-07-29T14:05:55.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:55.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:55.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 10 monclient(hunting): my global_id is 20425 2023-07-29T14:05:55.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.169+0000 7f447100f640 1 -- 172.21.15.40:0/2497766228 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f446c0b7870 con 0x7f446c1445e0 2023-07-29T14:05:55.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.170+0000 7f4471810640 10 monclient(hunting): handle_auth_done global_id 16436 payload 274 2023-07-29T14:05:55.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.170+0000 7f4471810640 1 -- 172.21.15.40:0/2497766228 >> v1:172.21.15.45:6789/0 conn(0x7f446c1445e0 legacy=0x7f446c144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.170+0000 7f4471810640 1 -- 172.21.15.40:0/2497766228 >> v1:172.21.15.40:6789/0 conn(0x7f446c0b74a0 legacy=0x7f446c13dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.170+0000 7f4471810640 10 monclient: _finish_hunting 0 2023-07-29T14:05:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.170+0000 7f4471810640 1 monclient: found mon.noname-b 2023-07-29T14:05:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.170+0000 7f4471810640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.170+0000 7f4471810640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:05:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.170+0000 7f4471810640 1 -- 172.21.15.40:0/2497766228 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0a9770 con 0x7f446c13e4f0 2023-07-29T14:05:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.172+0000 7f4471810640 1 --2- 172.21.15.40:0/2497766228 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c13e4f0 0x7f446c0aa150 secure :-1 s=READY pgs=3241 cs=0 l=1 rev1=1 crypto rx=0x7f445c0097d0 tx=0x7f445c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=272b761e90026bcb server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 1 -- 172.21.15.40:0/2497766228 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c031800 con 0x7f446c13e4f0 2023-07-29T14:05:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 10 monclient: dump: 2023-07-29T14:05:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:55.185 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:55.185 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:55.186 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 10 monclient: _finish_auth 0 2023-07-29T14:05:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 10 monclient: _check_auth_tickets 2023-07-29T14:05:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 1 -- 172.21.15.40:0/2497766228 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c031960 con 0x7f446c13e4f0 2023-07-29T14:05:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f4474275640 5 monclient: authenticate success, global_id 16436 2023-07-29T14:05:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 1 -- 172.21.15.40:0/2497766228 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c031cf0 con 0x7f446c13e4f0 2023-07-29T14:05:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:05:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f447100f640 10 monclient: dump: 2023-07-29T14:05:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:55.188 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f4474275640 10 monclient: get_monmap_and_config success 2023-07-29T14:05:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.175+0000 7f4474275640 4 set_mon_vals no callback set 2023-07-29T14:05:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.176+0000 7f4474275640 10 monclient: shutdown 2023-07-29T14:05:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.176+0000 7f4474275640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.176+0000 7f4474275640 1 -- 172.21.15.40:0/2497766228 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c13e4f0 msgr2=0x7f446c0aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.176+0000 7f4474275640 1 --2- 172.21.15.40:0/2497766228 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c13e4f0 0x7f446c0aa150 secure :-1 s=READY pgs=3241 cs=0 l=1 rev1=1 crypto rx=0x7f445c0097d0 tx=0x7f445c02b8a0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.177+0000 7f4474275640 1 -- 172.21.15.40:0/2497766228 shutdown_connections 2023-07-29T14:05:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.177+0000 7f4474275640 1 --2- 172.21.15.40:0/2497766228 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c13e4f0 0x7f446c0aa150 unknown :-1 s=CLOSED pgs=3241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.177+0000 7f4474275640 1 -- 172.21.15.40:0/2497766228 >> 172.21.15.40:0/2497766228 conn(0x7f446c065b00 msgr2=0x7f446c065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.177+0000 7f4474275640 1 -- 172.21.15.40:0/2497766228 shutdown_connections 2023-07-29T14:05:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.177+0000 7f4474275640 1 -- 172.21.15.40:0/2497766228 wait complete. 2023-07-29T14:05:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.177+0000 7f4474275640 10 monclient: build_initial_monmap 2023-07-29T14:05:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.177+0000 7f4474275640 10 monclient: monmap: 2023-07-29T14:05:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:05:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:05:55.191 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:05:55.191 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:05:55.191 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:05:55.191 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:55.191 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:05:55.191 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:05:55.191 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:05:55.191 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:55.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 librados: starting msgr at 2023-07-29T14:05:55.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 librados: starting objecter 2023-07-29T14:05:55.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 Processor -- start 2023-07-29T14:05:55.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 -- start start 2023-07-29T14:05:55.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 librados: setting wanted keys 2023-07-29T14:05:55.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 librados: calling monclient init 2023-07-29T14:05:55.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 10 monclient: init 2023-07-29T14:05:55.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 10 monclient: _reopen_session rank -1 2023-07-29T14:05:55.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T14:05:55.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c0b74a0 0x7f446c141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:55.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 10 monclient(hunting): picked mon.noname-c con 0x7f446c0b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:05:55.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c13e4f0 0x7f446c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:55.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 10 monclient(hunting): picked mon.noname-b con 0x7f446c13e4f0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:05:55.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c1445e0 0x7f446c0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:55.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 10 monclient(hunting): picked mon.noname-a con 0x7f446c1445e0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:05:55.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:55.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0b00c0 con 0x7f446c1445e0 2023-07-29T14:05:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0b0340 con 0x7f446c13e4f0 2023-07-29T14:05:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 10 monclient(hunting): start opening mon connection 2023-07-29T14:05:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c140680 con 0x7f446c0b74a0 2023-07-29T14:05:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 10 monclient(hunting): _renew_subs 2023-07-29T14:05:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.178+0000 7f4474275640 10 monclient(hunting): authenticate will time out at 3677.310952s 2023-07-29T14:05:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472011640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c0b74a0 0x7f446c141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472812640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c1445e0 0x7f446c0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:55.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4471810640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c13e4f0 0x7f446c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:55.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472011640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c0b74a0 0x7f446c141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:56782/0 (socket says 172.21.15.40:56782) 2023-07-29T14:05:55.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472011640 1 -- 172.21.15.40:0/448817344 learned_addr learned my addr 172.21.15.40:0/448817344 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:05:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472011640 10 monclient(hunting): get_auth_request con 0x7f446c0b74a0 auth_method 0 2023-07-29T14:05:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472011640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472011640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472011640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472812640 10 monclient(hunting): get_auth_request con 0x7f446c1445e0 auth_method 0 2023-07-29T14:05:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472812640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472812640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4472812640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4471810640 10 monclient(hunting): get_auth_request con 0x7f446c13e4f0 auth_method 0 2023-07-29T14:05:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4471810640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:05:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4471810640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:05:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.179+0000 7f4471810640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:05:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.180+0000 7f4472011640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.180+0000 7f4472011640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.180+0000 7f4472011640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.180+0000 7f4471810640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.180+0000 7f4471810640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.180+0000 7f4471810640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.180+0000 7f4472812640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:05:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.180+0000 7f4472812640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:05:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.180+0000 7f4472812640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:05:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.183+0000 7f4471810640 10 monclient(hunting): handle_auth_done global_id 20428 payload 824 2023-07-29T14:05:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.183+0000 7f4471810640 1 -- 172.21.15.40:0/448817344 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c0b74a0 msgr2=0x7f446c141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.183+0000 7f4471810640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c0b74a0 0x7f446c141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.183+0000 7f4471810640 1 -- 172.21.15.40:0/448817344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c1445e0 msgr2=0x7f446c0b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.183+0000 7f4471810640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c1445e0 0x7f446c0b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.183+0000 7f4471810640 10 monclient: _finish_hunting 0 2023-07-29T14:05:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.183+0000 7f4471810640 1 monclient: found mon.noname-b 2023-07-29T14:05:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.183+0000 7f4471810640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:05:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.183+0000 7f4471810640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.45:3300/0 2023-07-29T14:05:55.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.183+0000 7f4471810640 1 -- 172.21.15.40:0/448817344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c009400 con 0x7f446c13e4f0 2023-07-29T14:05:55.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.185+0000 7f4471810640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c13e4f0 0x7f446c0ae410 secure :-1 s=READY pgs=4794 cs=0 l=1 rev1=1 crypto rx=0x7f445c033000 tx=0x7f445c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.185+0000 7f4462ffd640 10 client.?.objecter ms_handle_connect 0x7f446c13e4f0 2023-07-29T14:05:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.185+0000 7f4462ffd640 10 client.?.objecter resend_mon_ops 2023-07-29T14:05:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 1 -- 172.21.15.40:0/448817344 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c03bcb0 con 0x7f446c13e4f0 2023-07-29T14:05:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:05:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 10 monclient: dump: 2023-07-29T14:05:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:55.205 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 10 monclient: _finish_auth 0 2023-07-29T14:05:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 10 monclient: _check_auth_tickets 2023-07-29T14:05:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:05:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 1 -- 172.21.15.40:0/448817344 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c03be50 con 0x7f446c13e4f0 2023-07-29T14:05:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:05:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 1 -- 172.21.15.40:0/448817344 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c045630 con 0x7f446c13e4f0 2023-07-29T14:05:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4474275640 5 monclient: authenticate success, global_id 20428 2023-07-29T14:05:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4474275640 10 monclient: _renew_subs 2023-07-29T14:05:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4474275640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T14:05:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4474275640 1 -- 172.21.15.40:0/448817344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446c0ae950 con 0x7f446c13e4f0 2023-07-29T14:05:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f447080e640 4 set_mon_vals no callback set 2023-07-29T14:05:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:05:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-07-29T14:05:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4462ffd640 10 monclient: dump: 2023-07-29T14:05:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:05:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:05:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:05:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:05:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:05:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:05:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:05:55.208 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:05:55.208 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:05:55.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4474275640 10 client.20428.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:05:55.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4474275640 10 monclient: _renew_subs 2023-07-29T14:05:55.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4474275640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T14:05:55.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4474275640 1 -- 172.21.15.40:0/448817344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446c0b0f80 con 0x7f446c13e4f0 2023-07-29T14:05:55.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.189+0000 7f4474275640 1 librados: init done 2023-07-29T14:05:55.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.190+0000 7f4474275640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:55.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.190+0000 7f4474275640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:55.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.190+0000 7f4474275640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.45:3300/0 2023-07-29T14:05:55.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.190+0000 7f4474275640 1 -- 172.21.15.40:0/448817344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446c145110 con 0x7f446c13e4f0 2023-07-29T14:05:55.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.210+0000 7f4462ffd640 1 -- 172.21.15.40:0/448817344 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f445c04e450 con 0x7f446c13e4f0 2023-07-29T14:05:55.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.210+0000 7f4462ffd640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f445006bf60 0x7f445006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:05:55.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.210+0000 7f4472011640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f445006bf60 0x7f445006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:05:55.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.210+0000 7f4472011640 10 monclient: get_auth_request con 0x7f445006bf60 auth_method 0 2023-07-29T14:05:55.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.211+0000 7f4472011640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f445006bf60 0x7f445006e410 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4468007fd0 tx=0x7f4468009040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:05:55.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.211+0000 7f4462ffd640 10 client.20428.objecter ms_handle_connect 0x7f445006bf60 2023-07-29T14:05:55.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.452+0000 7f4462ffd640 1 -- 172.21.15.40:0/448817344 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f445c02e070 con 0x7f446c13e4f0 2023-07-29T14:05:55.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.452+0000 7f4462ffd640 10 client.20428.objecter ms_dispatch 0x7f446c000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:05:55.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.452+0000 7f4462ffd640 3 client.20428.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:05:55.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.452+0000 7f4462ffd640 3 client.20428.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:05:55.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.452+0000 7f4462ffd640 20 client.20428.objecter dump_active .. 0 homeless 2023-07-29T14:05:55.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.462+0000 7f4462ffd640 1 -- 172.21.15.40:0/448817344 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f445c076510 con 0x7f446c13e4f0 2023-07-29T14:05:55.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.462+0000 7f4462ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:05:55.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.462+0000 7f4462ffd640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:05:55.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.643+0000 7f4474275640 1 -- 172.21.15.40:0/448817344 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f446c0aa750 con 0x7f445006bf60 2023-07-29T14:05:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.648+0000 7f4462ffd640 1 -- 172.21.15.40:0/448817344 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+151151 (secure 0 0 0) 0x7f446800f8e0 con 0x7f445006bf60 2023-07-29T14:05:55.649 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T14:05:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-29T14:05:55.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.650+0000 7f4474275640 10 librados: watch_flush enter 2023-07-29T14:05:55.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 10 librados: watch_flush exit 2023-07-29T14:05:55.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 20 client.20428.objecter shutdown clearing up homeless session... 2023-07-29T14:05:55.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 10 client.20428.objecter successfully canceled tick 2023-07-29T14:05:55.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 -- 172.21.15.40:0/448817344 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f445006bf60 msgr2=0x7f445006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:55.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f445006bf60 0x7f445006e410 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4468007fd0 tx=0x7f4468009040 comp rx=0 tx=0).stop 2023-07-29T14:05:55.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 10 monclient: shutdown 2023-07-29T14:05:55.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:05:55.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 -- 172.21.15.40:0/448817344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c13e4f0 msgr2=0x7f446c0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:05:55.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c13e4f0 0x7f446c0ae410 secure :-1 s=READY pgs=4794 cs=0 l=1 rev1=1 crypto rx=0x7f445c033000 tx=0x7f445c0046c0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 -- 172.21.15.40:0/448817344 shutdown_connections 2023-07-29T14:05:55.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f445006bf60 0x7f445006e410 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c13e4f0 0x7f446c0ae410 unknown :-1 s=CLOSED pgs=4794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c1445e0 0x7f446c0b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 --2- 172.21.15.40:0/448817344 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f446c0b74a0 0x7f446c141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:05:55.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 -- 172.21.15.40:0/448817344 >> 172.21.15.40:0/448817344 conn(0x7f446c065b00 msgr2=0x7f446c0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:05:55.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 -- 172.21.15.40:0/448817344 shutdown_connections 2023-07-29T14:05:55.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 -- 172.21.15.40:0/448817344 wait complete. 2023-07-29T14:05:55.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:05:55.651+0000 7f4474275640 1 librados: shutdown 2023-07-29T14:05:55.667 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":2139,"stamp":"2023-07-29T14:05:55.632521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":521884,"kb_used_data":50744,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754452836,"statfs":{"total":773094113280,"available":772559704064,"internally_reserved":0,"allocated":51961856,"data_stored":28370954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002655"},"pg_stats":[{"pgid":"7.1b","version":"0'0","reported_seq":970,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.822035+0000","last_change":"2023-07-29T13:56:55.662556+0000","last_active":"2023-07-29T14:05:39.822035+0000","last_peered":"2023-07-29T14:05:39.822035+0000","last_clean":"2023-07-29T14:05:39.822035+0000","last_became_active":"2023-07-29T13:56:55.656930+0000","last_became_peered":"2023-07-29T13:56:55.656930+0000","last_unstale":"2023-07-29T14:05:39.822035+0000","last_undegraded":"2023-07-29T14:05:39.822035+0000","last_fullsized":"2023-07-29T14:05:39.822035+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":979,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.638833+0000","last_change":"2023-07-29T13:41:38.328763+0000","last_active":"2023-07-29T14:05:39.638833+0000","last_peered":"2023-07-29T14:05:39.638833+0000","last_clean":"2023-07-29T14:05:39.638833+0000","last_became_active":"2023-07-29T13:41:38.322679+0000","last_became_peered":"2023-07-29T13:41:38.322679+0000","last_unstale":"2023-07-29T14:05:39.638833+0000","last_undegraded":"2023-07-29T14:05:39.638833+0000","last_fullsized":"2023-07-29T14:05:39.638833+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":967,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:51.506602+0000","last_change":"2023-07-29T13:29:52.105383+0000","last_active":"2023-07-29T14:05:51.506602+0000","last_peered":"2023-07-29T14:05:51.506602+0000","last_clean":"2023-07-29T14:05:51.506602+0000","last_became_active":"2023-07-29T13:29:52.099163+0000","last_became_peered":"2023-07-29T13:29:52.099163+0000","last_unstale":"2023-07-29T14:05:51.506602+0000","last_undegraded":"2023-07-29T14:05:51.506602+0000","last_fullsized":"2023-07-29T14:05:51.506602+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":933,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:42.595814+0000","last_change":"2023-07-29T13:29:41.816664+0000","last_active":"2023-07-29T14:05:42.595814+0000","last_peered":"2023-07-29T14:05:42.595814+0000","last_clean":"2023-07-29T14:05:42.595814+0000","last_became_active":"2023-07-29T13:29:41.782506+0000","last_became_peered":"2023-07-29T13:29:41.782506+0000","last_unstale":"2023-07-29T14:05:42.595814+0000","last_undegraded":"2023-07-29T14:05:42.595814+0000","last_fullsized":"2023-07-29T14:05:42.595814+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":960,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:49.198542+0000","last_change":"2023-07-29T14:05:49.205817+0000","last_active":"2023-07-29T14:05:49.198542+0000","last_peered":"2023-07-29T14:05:49.198542+0000","last_clean":"2023-07-29T14:05:49.198542+0000","last_became_active":"2023-07-29T14:05:38.060277+0000","last_became_peered":"2023-07-29T14:05:38.060277+0000","last_unstale":"2023-07-29T14:05:49.198542+0000","last_undegraded":"2023-07-29T14:05:49.198542+0000","last_fullsized":"2023-07-29T14:05:49.198542+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:49.197193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:49.197193+0000","last_clean_scrub_stamp":"2023-07-29T14:05:49.197193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:55:48.982266+0000","scrub_duration":0.090687935999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":915,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.803907+0000","last_change":"2023-07-29T13:53:19.606247+0000","last_active":"2023-07-29T14:05:39.803907+0000","last_peered":"2023-07-29T14:05:39.803907+0000","last_clean":"2023-07-29T14:05:39.803907+0000","last_became_active":"2023-07-29T13:53:19.600386+0000","last_became_peered":"2023-07-29T13:53:19.600386+0000","last_unstale":"2023-07-29T14:05:39.803907+0000","last_undegraded":"2023-07-29T14:05:39.803907+0000","last_fullsized":"2023-07-29T14:05:39.803907+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:46:03.055229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":937,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.571721+0000","last_change":"2023-07-29T13:56:55.406251+0000","last_active":"2023-07-29T14:05:37.571721+0000","last_peered":"2023-07-29T14:05:37.571721+0000","last_clean":"2023-07-29T14:05:37.571721+0000","last_became_active":"2023-07-29T13:56:55.400591+0000","last_became_peered":"2023-07-29T13:56:55.400591+0000","last_unstale":"2023-07-29T14:05:37.571721+0000","last_undegraded":"2023-07-29T14:05:37.571721+0000","last_fullsized":"2023-07-29T14:05:37.571721+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:40:48.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":956,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.933677+0000","last_change":"2023-07-29T13:29:31.089676+0000","last_active":"2023-07-29T14:05:38.933677+0000","last_peered":"2023-07-29T14:05:38.933677+0000","last_clean":"2023-07-29T14:05:38.933677+0000","last_became_active":"2023-07-29T13:29:31.072042+0000","last_became_peered":"2023-07-29T13:29:31.072042+0000","last_unstale":"2023-07-29T14:05:38.933677+0000","last_undegraded":"2023-07-29T14:05:38.933677+0000","last_fullsized":"2023-07-29T14:05:38.933677+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":914,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.798296+0000","last_change":"2023-07-29T13:53:19.677067+0000","last_active":"2023-07-29T14:05:39.798296+0000","last_peered":"2023-07-29T14:05:39.798296+0000","last_clean":"2023-07-29T14:05:39.798296+0000","last_became_active":"2023-07-29T13:53:19.671215+0000","last_became_peered":"2023-07-29T13:53:19.671215+0000","last_unstale":"2023-07-29T14:05:39.798296+0000","last_undegraded":"2023-07-29T14:05:39.798296+0000","last_fullsized":"2023-07-29T14:05:39.798296+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:51:18.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":988,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.398233+0000","last_change":"2023-07-29T13:56:55.517147+0000","last_active":"2023-07-29T14:05:38.398233+0000","last_peered":"2023-07-29T14:05:38.398233+0000","last_clean":"2023-07-29T14:05:38.398233+0000","last_became_active":"2023-07-29T13:56:55.511490+0000","last_became_peered":"2023-07-29T13:56:55.511490+0000","last_unstale":"2023-07-29T14:05:38.398233+0000","last_undegraded":"2023-07-29T14:05:38.398233+0000","last_fullsized":"2023-07-29T14:05:38.398233+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":970,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.921597+0000","last_change":"2023-07-29T14:05:37.627583+0000","last_active":"2023-07-29T14:05:38.921597+0000","last_peered":"2023-07-29T14:05:38.921597+0000","last_clean":"2023-07-29T14:05:38.921597+0000","last_became_active":"2023-07-29T14:05:37.617822+0000","last_became_peered":"2023-07-29T14:05:37.617822+0000","last_unstale":"2023-07-29T14:05:38.921597+0000","last_undegraded":"2023-07-29T14:05:38.921597+0000","last_fullsized":"2023-07-29T14:05:38.921597+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":914,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.555235+0000","last_change":"2023-07-29T13:53:19.644608+0000","last_active":"2023-07-29T14:05:39.555235+0000","last_peered":"2023-07-29T14:05:39.555235+0000","last_clean":"2023-07-29T14:05:39.555235+0000","last_became_active":"2023-07-29T13:53:19.637473+0000","last_became_peered":"2023-07-29T13:53:19.637473+0000","last_unstale":"2023-07-29T14:05:39.555235+0000","last_undegraded":"2023-07-29T14:05:39.555235+0000","last_fullsized":"2023-07-29T14:05:39.555235+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:53:02.384899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":934,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.800151+0000","last_change":"2023-07-29T13:29:33.530600+0000","last_active":"2023-07-29T14:05:39.800151+0000","last_peered":"2023-07-29T14:05:39.800151+0000","last_clean":"2023-07-29T14:05:39.800151+0000","last_became_active":"2023-07-29T13:29:33.515255+0000","last_became_peered":"2023-07-29T13:29:33.515255+0000","last_unstale":"2023-07-29T14:05:39.800151+0000","last_undegraded":"2023-07-29T14:05:39.800151+0000","last_fullsized":"2023-07-29T14:05:39.800151+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":987,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.667744+0000","last_change":"2023-07-29T14:05:37.517334+0000","last_active":"2023-07-29T14:05:39.667744+0000","last_peered":"2023-07-29T14:05:39.667744+0000","last_clean":"2023-07-29T14:05:39.667744+0000","last_became_active":"2023-07-29T14:05:37.510437+0000","last_became_peered":"2023-07-29T14:05:37.510437+0000","last_unstale":"2023-07-29T14:05:39.667744+0000","last_undegraded":"2023-07-29T14:05:39.667744+0000","last_fullsized":"2023-07-29T14:05:39.667744+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":939,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.573766+0000","last_change":"2023-07-29T13:56:55.146603+0000","last_active":"2023-07-29T14:05:37.573766+0000","last_peered":"2023-07-29T14:05:37.573766+0000","last_clean":"2023-07-29T14:05:37.573766+0000","last_became_active":"2023-07-29T13:56:55.139167+0000","last_became_peered":"2023-07-29T13:56:55.139167+0000","last_unstale":"2023-07-29T14:05:37.573766+0000","last_undegraded":"2023-07-29T14:05:37.573766+0000","last_fullsized":"2023-07-29T14:05:37.573766+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:56:23.477235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":953,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:52.412095+0000","last_change":"2023-07-29T14:05:37.621178+0000","last_active":"2023-07-29T14:05:52.412095+0000","last_peered":"2023-07-29T14:05:52.412095+0000","last_clean":"2023-07-29T14:05:52.412095+0000","last_became_active":"2023-07-29T14:05:37.614802+0000","last_became_peered":"2023-07-29T14:05:37.614802+0000","last_unstale":"2023-07-29T14:05:52.412095+0000","last_undegraded":"2023-07-29T14:05:52.412095+0000","last_fullsized":"2023-07-29T14:05:52.412095+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":991,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:42.239744+0000","last_change":"2023-07-29T13:56:55.073408+0000","last_active":"2023-07-29T14:05:42.239744+0000","last_peered":"2023-07-29T14:05:42.239744+0000","last_clean":"2023-07-29T14:05:42.239744+0000","last_became_active":"2023-07-29T13:56:55.065724+0000","last_became_peered":"2023-07-29T13:56:55.065724+0000","last_unstale":"2023-07-29T14:05:42.239744+0000","last_undegraded":"2023-07-29T14:05:42.239744+0000","last_fullsized":"2023-07-29T14:05:42.239744+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":973,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.656144+0000","last_change":"2023-07-29T13:28:14.434312+0000","last_active":"2023-07-29T14:05:39.656144+0000","last_peered":"2023-07-29T14:05:39.656144+0000","last_clean":"2023-07-29T14:05:39.656144+0000","last_became_active":"2023-07-29T13:28:14.423044+0000","last_became_peered":"2023-07-29T13:28:14.423044+0000","last_unstale":"2023-07-29T14:05:39.656144+0000","last_undegraded":"2023-07-29T14:05:39.656144+0000","last_fullsized":"2023-07-29T14:05:39.656144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1","version":"1004'8","reported_seq":997,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:52.065768+0000","last_change":"2023-07-29T14:05:52.069559+0000","last_active":"2023-07-29T14:05:52.065768+0000","last_peered":"2023-07-29T14:05:52.065768+0000","last_clean":"2023-07-29T14:05:52.065768+0000","last_became_active":"2023-07-29T13:41:38.521847+0000","last_became_peered":"2023-07-29T13:41:38.521847+0000","last_unstale":"2023-07-29T14:05:52.065768+0000","last_undegraded":"2023-07-29T14:05:52.065768+0000","last_fullsized":"2023-07-29T14:05:52.065768+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"1004'8","last_scrub_stamp":"2023-07-29T14:05:52.064254+0000","last_deep_scrub":"1004'8","last_deep_scrub_stamp":"2023-07-29T14:05:52.064254+0000","last_clean_scrub_stamp":"2023-07-29T14:05:52.064254+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T01:31:35.837707+0000","scrub_duration":0.082755203999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":945,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.566657+0000","last_change":"2023-07-29T13:56:55.263959+0000","last_active":"2023-07-29T14:05:37.566657+0000","last_peered":"2023-07-29T14:05:37.566657+0000","last_clean":"2023-07-29T14:05:37.566657+0000","last_became_active":"2023-07-29T13:56:55.258241+0000","last_became_peered":"2023-07-29T13:56:55.258241+0000","last_unstale":"2023-07-29T14:05:37.566657+0000","last_undegraded":"2023-07-29T14:05:37.566657+0000","last_fullsized":"2023-07-29T14:05:37.566657+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:01:42.517469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":968,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.670011+0000","last_change":"2023-07-29T13:29:52.230014+0000","last_active":"2023-07-29T14:05:39.670011+0000","last_peered":"2023-07-29T14:05:39.670011+0000","last_clean":"2023-07-29T14:05:39.670011+0000","last_became_active":"2023-07-29T13:29:52.224467+0000","last_became_peered":"2023-07-29T13:29:52.224467+0000","last_unstale":"2023-07-29T14:05:39.670011+0000","last_undegraded":"2023-07-29T14:05:39.670011+0000","last_fullsized":"2023-07-29T14:05:39.670011+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":975,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:42.481125+0000","last_change":"2023-07-29T13:28:14.652586+0000","last_active":"2023-07-29T14:05:42.481125+0000","last_peered":"2023-07-29T14:05:42.481125+0000","last_clean":"2023-07-29T14:05:42.481125+0000","last_became_active":"2023-07-29T13:28:14.642023+0000","last_became_peered":"2023-07-29T13:28:14.642023+0000","last_unstale":"2023-07-29T14:05:42.481125+0000","last_undegraded":"2023-07-29T14:05:42.481125+0000","last_fullsized":"2023-07-29T14:05:42.481125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":935,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.806868+0000","last_change":"2023-07-29T13:29:41.869711+0000","last_active":"2023-07-29T14:05:39.806868+0000","last_peered":"2023-07-29T14:05:39.806868+0000","last_clean":"2023-07-29T14:05:39.806868+0000","last_became_active":"2023-07-29T13:29:41.844314+0000","last_became_peered":"2023-07-29T13:29:41.844314+0000","last_unstale":"2023-07-29T14:05:39.806868+0000","last_undegraded":"2023-07-29T14:05:39.806868+0000","last_fullsized":"2023-07-29T14:05:39.806868+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":964,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.476599+0000","last_change":"2023-07-29T13:56:55.159956+0000","last_active":"2023-07-29T14:05:37.476599+0000","last_peered":"2023-07-29T14:05:37.476599+0000","last_clean":"2023-07-29T14:05:37.476599+0000","last_became_active":"2023-07-29T13:56:55.150514+0000","last_became_peered":"2023-07-29T13:56:55.150514+0000","last_unstale":"2023-07-29T14:05:37.476599+0000","last_undegraded":"2023-07-29T14:05:37.476599+0000","last_fullsized":"2023-07-29T14:05:37.476599+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:16:38.329000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1007,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.654218+0000","last_change":"2023-07-29T13:56:55.615681+0000","last_active":"2023-07-29T14:05:39.654218+0000","last_peered":"2023-07-29T14:05:39.654218+0000","last_clean":"2023-07-29T14:05:39.654218+0000","last_became_active":"2023-07-29T13:56:55.609414+0000","last_became_peered":"2023-07-29T13:56:55.609414+0000","last_unstale":"2023-07-29T14:05:39.654218+0000","last_undegraded":"2023-07-29T14:05:39.654218+0000","last_fullsized":"2023-07-29T14:05:39.654218+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1025,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:52.312489+0000","last_change":"2023-07-29T14:05:52.318168+0000","last_active":"2023-07-29T14:05:52.312489+0000","last_peered":"2023-07-29T14:05:52.312489+0000","last_clean":"2023-07-29T14:05:52.312489+0000","last_became_active":"2023-07-29T13:56:55.543861+0000","last_became_peered":"2023-07-29T13:56:55.543861+0000","last_unstale":"2023-07-29T14:05:52.312489+0000","last_undegraded":"2023-07-29T14:05:52.312489+0000","last_fullsized":"2023-07-29T14:05:52.312489+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:52.310408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:52.310408+0000","last_clean_scrub_stamp":"2023-07-29T14:05:52.310408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:37:08.149976+0000","scrub_duration":0.039706038999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":951,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.620825+0000","last_change":"2023-07-29T14:05:37.613527+0000","last_active":"2023-07-29T14:05:37.620825+0000","last_peered":"2023-07-29T14:05:37.620825+0000","last_clean":"2023-07-29T14:05:37.620825+0000","last_became_active":"2023-07-29T14:05:37.607023+0000","last_became_peered":"2023-07-29T14:05:37.607023+0000","last_unstale":"2023-07-29T14:05:37.620825+0000","last_undegraded":"2023-07-29T14:05:37.620825+0000","last_fullsized":"2023-07-29T14:05:37.620825+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:07:39.130065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":975,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:42.481735+0000","last_change":"2023-07-29T13:28:14.713485+0000","last_active":"2023-07-29T14:05:42.481735+0000","last_peered":"2023-07-29T14:05:42.481735+0000","last_clean":"2023-07-29T14:05:42.481735+0000","last_became_active":"2023-07-29T13:28:14.708042+0000","last_became_peered":"2023-07-29T13:28:14.708042+0000","last_unstale":"2023-07-29T14:05:42.481735+0000","last_undegraded":"2023-07-29T14:05:42.481735+0000","last_fullsized":"2023-07-29T14:05:42.481735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:28:12.371049+0000","last_clean_scrub_stamp":"2023-07-29T13:28:12.371049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"1004'78","reported_seq":1116,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:51.159032+0000","last_change":"2023-07-29T14:05:51.166709+0000","last_active":"2023-07-29T14:05:51.159032+0000","last_peered":"2023-07-29T14:05:51.159032+0000","last_clean":"2023-07-29T14:05:51.159032+0000","last_became_active":"2023-07-29T13:56:55.157702+0000","last_became_peered":"2023-07-29T13:56:55.157702+0000","last_unstale":"2023-07-29T14:05:51.159032+0000","last_undegraded":"2023-07-29T14:05:51.159032+0000","last_fullsized":"2023-07-29T14:05:51.159032+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"1004'78","last_scrub_stamp":"2023-07-29T14:05:51.155533+0000","last_deep_scrub":"1004'78","last_deep_scrub_stamp":"2023-07-29T14:05:51.155533+0000","last_clean_scrub_stamp":"2023-07-29T14:05:51.155533+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:17:55.983069+0000","scrub_duration":0.143700576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1008,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:50.973540+0000","last_change":"2023-07-29T14:05:50.975646+0000","last_active":"2023-07-29T14:05:50.973540+0000","last_peered":"2023-07-29T14:05:50.973540+0000","last_clean":"2023-07-29T14:05:50.973540+0000","last_became_active":"2023-07-29T13:56:55.341426+0000","last_became_peered":"2023-07-29T13:56:55.341426+0000","last_unstale":"2023-07-29T14:05:50.973540+0000","last_undegraded":"2023-07-29T14:05:50.973540+0000","last_fullsized":"2023-07-29T14:05:50.973540+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:50.972716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:50.972716+0000","last_clean_scrub_stamp":"2023-07-29T14:05:50.972716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:38:24.126097+0000","scrub_duration":0.034645589999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":948,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.408470+0000","last_change":"2023-07-29T13:41:38.063359+0000","last_active":"2023-07-29T14:05:38.408470+0000","last_peered":"2023-07-29T14:05:38.408470+0000","last_clean":"2023-07-29T14:05:38.408470+0000","last_became_active":"2023-07-29T13:41:38.057649+0000","last_became_peered":"2023-07-29T13:41:38.057649+0000","last_unstale":"2023-07-29T14:05:38.408470+0000","last_undegraded":"2023-07-29T14:05:38.408470+0000","last_fullsized":"2023-07-29T14:05:38.408470+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":926,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.484786+0000","last_change":"2023-07-29T13:53:19.566408+0000","last_active":"2023-07-29T14:05:37.484786+0000","last_peered":"2023-07-29T14:05:37.484786+0000","last_clean":"2023-07-29T14:05:37.484786+0000","last_became_active":"2023-07-29T13:53:19.560480+0000","last_became_peered":"2023-07-29T13:53:19.560480+0000","last_unstale":"2023-07-29T14:05:37.484786+0000","last_undegraded":"2023-07-29T14:05:37.484786+0000","last_fullsized":"2023-07-29T14:05:37.484786+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:01:49.170231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":914,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.488215+0000","last_change":"2023-07-29T13:53:19.617225+0000","last_active":"2023-07-29T14:05:37.488215+0000","last_peered":"2023-07-29T14:05:37.488215+0000","last_clean":"2023-07-29T14:05:37.488215+0000","last_became_active":"2023-07-29T13:53:19.611477+0000","last_became_peered":"2023-07-29T13:53:19.611477+0000","last_unstale":"2023-07-29T14:05:37.488215+0000","last_undegraded":"2023-07-29T14:05:37.488215+0000","last_fullsized":"2023-07-29T14:05:37.488215+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:52:56.727655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":951,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.539929+0000","last_change":"2023-07-29T13:56:55.182600+0000","last_active":"2023-07-29T14:05:37.539929+0000","last_peered":"2023-07-29T14:05:37.539929+0000","last_clean":"2023-07-29T14:05:37.539929+0000","last_became_active":"2023-07-29T13:56:55.173228+0000","last_became_peered":"2023-07-29T13:56:55.173228+0000","last_unstale":"2023-07-29T14:05:37.539929+0000","last_undegraded":"2023-07-29T14:05:37.539929+0000","last_fullsized":"2023-07-29T14:05:37.539929+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:57:08.005958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":938,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.548791+0000","last_change":"2023-07-29T13:56:55.243737+0000","last_active":"2023-07-29T14:05:37.548791+0000","last_peered":"2023-07-29T14:05:37.548791+0000","last_clean":"2023-07-29T14:05:37.548791+0000","last_became_active":"2023-07-29T13:56:55.238036+0000","last_became_peered":"2023-07-29T13:56:55.238036+0000","last_unstale":"2023-07-29T14:05:37.548791+0000","last_undegraded":"2023-07-29T14:05:37.548791+0000","last_fullsized":"2023-07-29T14:05:37.548791+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:04:27.494041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":933,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.798441+0000","last_change":"2023-07-29T13:29:33.527250+0000","last_active":"2023-07-29T14:05:39.798441+0000","last_peered":"2023-07-29T14:05:39.798441+0000","last_clean":"2023-07-29T14:05:39.798441+0000","last_became_active":"2023-07-29T13:29:33.508113+0000","last_became_peered":"2023-07-29T13:29:33.508113+0000","last_unstale":"2023-07-29T14:05:39.798441+0000","last_undegraded":"2023-07-29T14:05:39.798441+0000","last_fullsized":"2023-07-29T14:05:39.798441+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":941,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.401769+0000","last_change":"2023-07-29T13:29:32.261353+0000","last_active":"2023-07-29T14:05:38.401769+0000","last_peered":"2023-07-29T14:05:38.401769+0000","last_clean":"2023-07-29T14:05:38.401769+0000","last_became_active":"2023-07-29T13:29:32.123788+0000","last_became_peered":"2023-07-29T13:29:32.123788+0000","last_unstale":"2023-07-29T14:05:38.401769+0000","last_undegraded":"2023-07-29T14:05:38.401769+0000","last_fullsized":"2023-07-29T14:05:38.401769+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:25:55.926424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":937,"reported_epoch":1003,"state":"active+clean","last_fresh":"2023-07-29T14:05:37.564990+0000","last_change":"2023-07-29T13:56:55.597264+0000","last_active":"2023-07-29T14:05:37.564990+0000","last_peered":"2023-07-29T14:05:37.564990+0000","last_clean":"2023-07-29T14:05:37.564990+0000","last_became_active":"2023-07-29T13:56:55.591631+0000","last_became_peered":"2023-07-29T13:56:55.591631+0000","last_unstale":"2023-07-29T14:05:37.564990+0000","last_undegraded":"2023-07-29T14:05:37.564990+0000","last_fullsized":"2023-07-29T14:05:37.564990+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:55:19.929908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1005,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.652371+0000","last_change":"2023-07-29T13:56:55.572081+0000","last_active":"2023-07-29T14:05:39.652371+0000","last_peered":"2023-07-29T14:05:39.652371+0000","last_clean":"2023-07-29T14:05:39.652371+0000","last_became_active":"2023-07-29T13:56:55.564155+0000","last_became_peered":"2023-07-29T13:56:55.564155+0000","last_unstale":"2023-07-29T14:05:39.652371+0000","last_undegraded":"2023-07-29T14:05:39.652371+0000","last_fullsized":"2023-07-29T14:05:39.652371+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":975,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:47.246847+0000","last_change":"2023-07-29T13:53:19.441193+0000","last_active":"2023-07-29T14:05:47.246847+0000","last_peered":"2023-07-29T14:05:47.246847+0000","last_clean":"2023-07-29T14:05:47.246847+0000","last_became_active":"2023-07-29T13:53:19.433950+0000","last_became_peered":"2023-07-29T13:53:19.433950+0000","last_unstale":"2023-07-29T14:05:47.246847+0000","last_undegraded":"2023-07-29T14:05:47.246847+0000","last_fullsized":"2023-07-29T14:05:47.246847+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":961,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:47.247527+0000","last_change":"2023-07-29T14:05:37.522505+0000","last_active":"2023-07-29T14:05:47.247527+0000","last_peered":"2023-07-29T14:05:47.247527+0000","last_clean":"2023-07-29T14:05:47.247527+0000","last_became_active":"2023-07-29T14:05:37.513918+0000","last_became_peered":"2023-07-29T14:05:37.513918+0000","last_unstale":"2023-07-29T14:05:47.247527+0000","last_undegraded":"2023-07-29T14:05:47.247527+0000","last_fullsized":"2023-07-29T14:05:47.247527+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":17,"seq":73014444475,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59672,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312168,"statfs":{"total":96636764160,"available":96575660032,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":15.343999999999999,"5min":16.634,"15min":170.43700000000001},"min":{"1min":6.3490000000000002,"5min":6.3330000000000002,"15min":4.7590000000000003},"max":{"1min":111.66,"5min":172.44399999999999,"15min":12751.485000000001},"last":7.7450000000000001},{"interface":"front","average":{"1min":16.254000000000001,"5min":16.774000000000001,"15min":173.22},"min":{"1min":6.79,"5min":5.4500000000000002,"15min":5.4500000000000002},"max":{"1min":129.28399999999999,"5min":173.48699999999999,"15min":12747.76},"last":7.0209999999999999}]},{"osd":1,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":18.536000000000001,"5min":1543.261,"15min":525.65800000000002},"min":{"1min":7.0010000000000003,"5min":6.5149999999999997,"15min":6.5149999999999997},"max":{"1min":99.557000000000002,"5min":29038.089,"15min":29038.089},"last":13.1},{"interface":"front","average":{"1min":22.518000000000001,"5min":1548.174,"15min":530.21500000000003},"min":{"1min":6.9089999999999998,"5min":6.4749999999999996,"15min":6.2249999999999996},"max":{"1min":102.327,"5min":29039.032999999999,"15min":29039.032999999999},"last":11.202}]},{"osd":2,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":23.297000000000001,"5min":21.439,"15min":20.452000000000002},"min":{"1min":7.992,"5min":7.3200000000000003,"15min":7.3200000000000003},"max":{"1min":104.566,"5min":249.84800000000001,"15min":784.66200000000003},"last":10.622999999999999},{"interface":"front","average":{"1min":21.670000000000002,"5min":21.084,"15min":19.474},"min":{"1min":8.6280000000000001,"5min":8.3919999999999995,"15min":7.8049999999999997},"max":{"1min":99.706000000000003,"5min":202.64500000000001,"15min":368.62200000000001},"last":13.683}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.91,"5min":25.608000000000001,"15min":478.83800000000002},"min":{"1min":7.0910000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"max":{"1min":101.714,"5min":312.73599999999999,"15min":28381.707999999999},"last":10.063000000000001},{"interface":"front","average":{"1min":20.298999999999999,"5min":27.803999999999998,"15min":474.06299999999999},"min":{"1min":9.3219999999999992,"5min":9.3219999999999992,"15min":8.7520000000000007},"max":{"1min":97.727999999999994,"5min":242.63,"15min":28126.012999999999},"last":20.103999999999999}]},{"osd":5,"last update":"Sat Jul 29 14:05:01 2023","interfaces":[{"interface":"back","average":{"1min":17.222999999999999,"5min":20.141999999999999,"15min":32.993000000000002},"min":{"1min":10.452,"5min":8.4540000000000006,"15min":7.3689999999999998},"max":{"1min":79.867000000000004,"5min":85.578999999999994,"15min":488.72300000000001},"last":14.365},{"interface":"front","average":{"1min":14.161,"5min":19.236999999999998,"15min":21.838000000000001},"min":{"1min":8.1379999999999999,"5min":8.1379999999999999,"15min":7.3840000000000003},"max":{"1min":52.622,"5min":106.634,"15min":128.791},"last":12.542999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":24.904,"5min":20.061,"15min":22.629999999999999},"min":{"1min":9.1470000000000002,"5min":8.0129999999999999,"15min":7.7779999999999996},"max":{"1min":106.33199999999999,"5min":124.819,"15min":787.18499999999995},"last":11.845000000000001},{"interface":"front","average":{"1min":25.198,"5min":20.042000000000002,"15min":22.454999999999998},"min":{"1min":9.8010000000000002,"5min":8.5790000000000006,"15min":8.3170000000000002},"max":{"1min":112.328,"5min":112.328,"15min":646.87599999999998},"last":18.960000000000001}]},{"osd":7,"last update":"Sat Jul 29 14:04:50 2023","interfaces":[{"interface":"back","average":{"1min":27.215,"5min":22.292000000000002,"15min":48.816000000000003},"min":{"1min":11.487,"5min":9.2620000000000005,"15min":9.2620000000000005},"max":{"1min":149.43199999999999,"5min":149.43199999999999,"15min":648.54200000000003},"last":14.936999999999999},{"interface":"front","average":{"1min":23.321000000000002,"5min":20.568999999999999,"15min":38.241999999999997},"min":{"1min":10.385,"5min":9.2300000000000004,"15min":8.7309999999999999},"max":{"1min":75.954999999999998,"5min":75.954999999999998,"15min":364.86500000000001},"last":15.615}]}]},{"osd":2,"up_from":17,"seq":73014444487,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63536,"kb_used_data":6248,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94308304,"statfs":{"total":96636764160,"available":96571703296,"internally_reserved":0,"allocated":6397952,"data_stored":3446913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":13.247999999999999,"15min":130.82599999999999},"min":{"1min":5.3789999999999996,"5min":5.0209999999999999,"15min":4.476},"max":{"1min":29.634,"5min":93.590999999999994,"15min":10009.493},"last":7.7279999999999998},{"interface":"front","average":{"1min":13.936999999999999,"5min":13.526,"15min":131.76599999999999},"min":{"1min":7.1299999999999999,"5min":6.2069999999999999,"15min":5.617},"max":{"1min":37.527999999999999,"5min":99.156999999999996,"15min":9995.5869999999995},"last":14.430999999999999}]},{"osd":1,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":16.233000000000001,"5min":1412.377,"15min":481.59100000000001},"min":{"1min":6.7910000000000004,"5min":6.0899999999999999,"15min":6.0899999999999999},"max":{"1min":101.836,"5min":29343.018,"15min":29343.018},"last":7.0940000000000003},{"interface":"front","average":{"1min":16.434000000000001,"5min":1411.5540000000001,"15min":479.24299999999999},"min":{"1min":5.7990000000000004,"5min":5.7990000000000004,"15min":5.7990000000000004},"max":{"1min":137.94,"5min":29342.263999999999,"15min":29342.263999999999},"last":9.3719999999999999}]},{"osd":3,"last update":"Sat Jul 29 14:05:44 2023","interfaces":[{"interface":"back","average":{"1min":13.795,"5min":19.425999999999998,"15min":365.00299999999999},"min":{"1min":8.1449999999999996,"5min":6.7809999999999997,"15min":6.7809999999999997},"max":{"1min":30.794,"5min":331.92399999999998,"15min":29770.313999999998},"last":13.282999999999999},{"interface":"front","average":{"1min":13.835000000000001,"5min":21.202000000000002,"15min":358.64400000000001},"min":{"1min":7.351,"5min":7.351,"15min":6.4749999999999996},"max":{"1min":34.841000000000001,"5min":332.49900000000002,"15min":29586.387999999999},"last":15.353999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:04:53 2023","interfaces":[{"interface":"back","average":{"1min":12.471,"5min":819.34000000000003,"15min":870.13199999999995},"min":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"max":{"1min":52.584000000000003,"5min":25685.629000000001,"15min":32071.659},"last":12.699},{"interface":"front","average":{"1min":15.125999999999999,"5min":814.09699999999998,"15min":870.976},"min":{"1min":8.8330000000000002,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":35.020000000000003,"5min":25610.863000000001,"15min":32126.150000000001},"last":18.972999999999999}]},{"osd":5,"last update":"Sat Jul 29 14:05:06 2023","interfaces":[{"interface":"back","average":{"1min":14.398,"5min":20.428999999999998,"15min":38.106999999999999},"min":{"1min":8.7579999999999991,"5min":8.4269999999999996,"15min":8.4269999999999996},"max":{"1min":18.667000000000002,"5min":193.45400000000001,"15min":428.04000000000002},"last":10.576000000000001},{"interface":"front","average":{"1min":12.019,"5min":20.446999999999999,"15min":41.822000000000003},"min":{"1min":6.891,"5min":6.891,"15min":6.891},"max":{"1min":19.634,"5min":174.292,"15min":430.15499999999997},"last":8.8059999999999992}]},{"osd":6,"last update":"Sat Jul 29 14:05:40 2023","interfaces":[{"interface":"back","average":{"1min":17.140000000000001,"5min":21.350999999999999,"15min":20.024000000000001},"min":{"1min":8.3450000000000006,"5min":8.3450000000000006,"15min":8.3450000000000006},"max":{"1min":77.855000000000004,"5min":318.45299999999997,"15min":318.45299999999997},"last":11.336},{"interface":"front","average":{"1min":17.119,"5min":21.058,"15min":20.518999999999998},"min":{"1min":9.2230000000000008,"5min":9.2230000000000008,"15min":7.5300000000000002},"max":{"1min":44.755000000000003,"5min":349.34899999999999,"15min":349.34899999999999},"last":16.219000000000001}]},{"osd":7,"last update":"Sat Jul 29 14:05:44 2023","interfaces":[{"interface":"back","average":{"1min":19.048999999999999,"5min":19.532,"15min":21.023},"min":{"1min":10.064,"5min":8.5079999999999991,"15min":7.5890000000000004},"max":{"1min":80.870000000000005,"5min":291.80900000000003,"15min":291.80900000000003},"last":16.82},{"interface":"front","average":{"1min":22.766999999999999,"5min":21.436,"15min":22.666},"min":{"1min":9.3379999999999992,"5min":9.1940000000000008,"15min":9.1940000000000008},"max":{"1min":82.373999999999995,"5min":291.44200000000001,"15min":291.44200000000001},"last":17.373999999999999}]}]},{"osd":1,"up_from":447,"seq":1919850381745,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59832,"kb_used_data":6640,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312008,"statfs":{"total":96636764160,"available":96575496192,"internally_reserved":0,"allocated":6799360,"data_stored":3844753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":20.931000000000001,"5min":20.015999999999998,"15min":112.807},"min":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.3680000000000003},"max":{"1min":116.512,"5min":364.47800000000001,"15min":12525.57},"last":15.739000000000001},{"interface":"front","average":{"1min":18.943000000000001,"5min":20.664000000000001,"15min":112.874},"min":{"1min":6.0970000000000004,"5min":6.0970000000000004,"15min":4.6820000000000004},"max":{"1min":100.402,"5min":372.00999999999999,"15min":12523.974},"last":14.885999999999999}]},{"osd":2,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.987,"5min":26.809000000000001,"15min":19.399999999999999},"min":{"1min":7.819,"5min":6.5110000000000001,"15min":5.5919999999999996},"max":{"1min":69.497,"5min":360.88099999999997,"15min":360.88099999999997},"last":12.493},{"interface":"front","average":{"1min":19.43,"5min":28.155999999999999,"15min":20.445},"min":{"1min":7.093,"5min":7.093,"15min":6.2729999999999997},"max":{"1min":93.792000000000002,"5min":368.51400000000001,"15min":368.51400000000001},"last":8.2720000000000002}]},{"osd":3,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":26.696000000000002,"5min":30.559000000000001,"15min":531.69799999999998},"min":{"1min":9.1590000000000007,"5min":6.7140000000000004,"15min":6.7140000000000004},"max":{"1min":216.16300000000001,"5min":435.92200000000003,"15min":34504.942000000003},"last":11.872},{"interface":"front","average":{"1min":31.332000000000001,"5min":41.100999999999999,"15min":539.97299999999996},"min":{"1min":10.423,"5min":10.423,"15min":5.3639999999999999},"max":{"1min":238.33699999999999,"5min":700.10500000000002,"15min":34620.970999999998},"last":20.712}]},{"osd":4,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":15.427,"5min":1170.991,"15min":980.98599999999999},"min":{"1min":8.4130000000000003,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":50.289000000000001,"5min":25247.098999999998,"15min":32258.529999999999},"last":12.82},{"interface":"front","average":{"1min":16.536000000000001,"5min":1165.587,"15min":967.39999999999998},"min":{"1min":11.340999999999999,"5min":8.9169999999999998,"15min":6.5090000000000003},"max":{"1min":29.376999999999999,"5min":25174.008999999998,"15min":31933.992999999999},"last":13.553000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:05:13 2023","interfaces":[{"interface":"back","average":{"1min":15.249000000000001,"5min":33.526000000000003,"15min":43.616},"min":{"1min":10.416,"5min":10.294,"15min":10.294},"max":{"1min":23.649999999999999,"5min":433.79300000000001,"15min":433.79300000000001},"last":18.530000000000001},{"interface":"front","average":{"1min":15.952,"5min":36.515000000000001,"15min":46.286000000000001},"min":{"1min":11.467000000000001,"5min":9.6929999999999996,"15min":8.7260000000000009},"max":{"1min":22.300999999999998,"5min":419.22800000000001,"15min":419.22800000000001},"last":16.413}]},{"osd":6,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":16.931999999999999,"5min":21.992000000000001,"15min":19.459},"min":{"1min":6.6079999999999997,"5min":6.327,"15min":6.327},"max":{"1min":154.92400000000001,"5min":357.20499999999998,"15min":357.20499999999998},"last":7.1299999999999999},{"interface":"front","average":{"1min":20.251999999999999,"5min":27.460999999999999,"15min":23.411000000000001},"min":{"1min":10.877000000000001,"5min":9.5079999999999991,"15min":9.5079999999999991},"max":{"1min":155.994,"5min":359.68099999999998,"15min":359.68099999999998},"last":11.031000000000001}]},{"osd":7,"last update":"Sat Jul 29 14:04:54 2023","interfaces":[{"interface":"back","average":{"1min":19.683,"5min":35.137999999999998,"15min":23.725999999999999},"min":{"1min":10.130000000000001,"5min":8.5199999999999996,"15min":7.5919999999999996},"max":{"1min":92.168999999999997,"5min":563.29700000000003,"15min":563.29700000000003},"last":19.414999999999999},{"interface":"front","average":{"1min":17.949000000000002,"5min":30.442,"15min":21.349},"min":{"1min":8.2240000000000002,"5min":8.2240000000000002,"15min":7.2919999999999998},"max":{"1min":113.15900000000001,"5min":352.65199999999999,"15min":352.65199999999999},"last":16.957999999999998}]}]},{"osd":0,"up_from":1002,"seq":4303557231031,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59436,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312404,"statfs":{"total":96636764160,"available":96575901696,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":71.099999999999994,"5min":1023.201,"15min":351.10899999999998},"min":{"1min":8.9689999999999994,"5min":5.9249999999999998,"15min":4.6269999999999998},"max":{"1min":350.00200000000001,"5min":27534,"15min":27534},"last":14.087999999999999},{"interface":"front","average":{"1min":99.875,"5min":1025.7080000000001,"15min":350.98899999999998},"min":{"1min":5.9589999999999996,"5min":5.7240000000000002,"15min":5.0739999999999998},"max":{"1min":591.69600000000003,"5min":27535.008999999998,"15min":27535.008999999998},"last":15.349}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.170999999999999,"5min":22.285,"15min":18.638000000000002},"min":{"1min":11.680999999999999,"5min":8.1069999999999993,"15min":6.6159999999999997},"max":{"1min":364.62799999999999,"5min":364.62799999999999,"15min":364.62799999999999},"last":8.0399999999999991},{"interface":"front","average":{"1min":45.890000000000001,"5min":26.417999999999999,"15min":21.443000000000001},"min":{"1min":14.172000000000001,"5min":10.023999999999999,"15min":8.8780000000000001},"max":{"1min":370.34300000000002,"5min":370.34300000000002,"15min":370.34300000000002},"last":14.726000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":66.832999999999998,"5min":25.178000000000001,"15min":355.89100000000002},"min":{"1min":6.7610000000000001,"5min":6.4409999999999998,"15min":5.891},"max":{"1min":444.59500000000003,"5min":444.59500000000003,"15min":30170.040000000001},"last":17.754999999999999},{"interface":"front","average":{"1min":65.754000000000005,"5min":33.606000000000002,"15min":363.07400000000001},"min":{"1min":10.266999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"max":{"1min":386.06099999999998,"5min":386.06099999999998,"15min":30171.194},"last":9.1159999999999997}]},{"osd":4,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":92.555000000000007,"5min":36.014000000000003,"15min":585.43899999999996},"min":{"1min":12.201000000000001,"5min":6.4480000000000004,"15min":6.0979999999999999},"max":{"1min":445.65300000000002,"5min":445.65300000000002,"15min":29892.669999999998},"last":15.955},{"interface":"front","average":{"1min":115.806,"5min":40.560000000000002,"15min":588.88900000000001},"min":{"1min":7.923,"5min":5.9400000000000004,"15min":5.9400000000000004},"max":{"1min":593.13499999999999,"5min":593.13499999999999,"15min":29877.289000000001},"last":11.327999999999999}]},{"osd":5,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.341000000000001,"5min":28.972000000000001,"15min":56.932000000000002},"min":{"1min":12.702,"5min":9.5700000000000003,"15min":9.5700000000000003},"max":{"1min":403.13999999999999,"5min":403.13999999999999,"15min":508.02100000000002},"last":13.364000000000001},{"interface":"front","average":{"1min":43.966999999999999,"5min":23.216000000000001,"15min":32.374000000000002},"min":{"1min":7.6429999999999998,"5min":7.1210000000000004,"15min":7.1210000000000004},"max":{"1min":590.58500000000004,"5min":590.58500000000004,"15min":590.58500000000004},"last":11.939}]},{"osd":6,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":52.365000000000002,"5min":30.853999999999999,"15min":25.227},"min":{"1min":16.876000000000001,"5min":10.409000000000001,"15min":9.9350000000000005},"max":{"1min":375.36500000000001,"5min":375.36500000000001,"15min":375.36500000000001},"last":10.189},{"interface":"front","average":{"1min":55.634,"5min":32.579999999999998,"15min":27.417999999999999},"min":{"1min":18.984999999999999,"5min":11.564,"15min":11.564},"max":{"1min":380.31599999999997,"5min":380.31599999999997,"15min":380.31599999999997},"last":18.462}]},{"osd":7,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":57.960999999999999,"5min":28.638000000000002,"15min":20.780000000000001},"min":{"1min":17.748999999999999,"5min":8.4939999999999998,"15min":7.9349999999999996},"max":{"1min":385.01900000000001,"5min":385.01900000000001,"15min":385.01900000000001},"last":19.370999999999999},{"interface":"front","average":{"1min":54.853999999999999,"5min":29.765999999999998,"15min":25.545000000000002},"min":{"1min":10.992000000000001,"5min":8.1229999999999993,"15min":8.1229999999999993},"max":{"1min":446.69099999999997,"5min":446.69099999999997,"15min":446.69099999999997},"last":12.792999999999999}]}]},{"osd":4,"up_from":16,"seq":68719477164,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59676,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312164,"statfs":{"total":96636764160,"available":96575655936,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":13.432,"5min":21.457000000000001,"15min":87.253},"min":{"1min":6.4029999999999996,"5min":4.6230000000000002,"15min":4.6230000000000002},"max":{"1min":48.048999999999999,"5min":495.86900000000003,"15min":10627.831},"last":9.5980000000000008},{"interface":"front","average":{"1min":18.010000000000002,"5min":25.013000000000002,"15min":90.852000000000004},"min":{"1min":7.2400000000000002,"5min":6.0990000000000002,"15min":5.6660000000000004},"max":{"1min":124.64100000000001,"5min":446.84100000000001,"15min":10653.877},"last":6.5179999999999998}]},{"osd":1,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":11.276999999999999,"5min":1441.567,"15min":495.98200000000003},"min":{"1min":7.2640000000000002,"5min":7.2640000000000002,"15min":5.9909999999999997},"max":{"1min":21.369,"5min":26872.857,"15min":26872.857},"last":12.369999999999999},{"interface":"front","average":{"1min":11.013,"5min":1440.0119999999999,"15min":493.40499999999997},"min":{"1min":7.4269999999999996,"5min":7.3090000000000002,"15min":5.6909999999999998},"max":{"1min":17.440999999999999,"5min":26874.048999999999,"15min":26874.048999999999},"last":15.836}]},{"osd":2,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":12.445,"5min":24.050999999999998,"15min":23.050000000000001},"min":{"1min":8.9000000000000004,"5min":7.3019999999999996,"15min":7.3019999999999996},"max":{"1min":15.904999999999999,"5min":453.40100000000001,"15min":597.11800000000005},"last":13.031000000000001},{"interface":"front","average":{"1min":15.029999999999999,"5min":27.077999999999999,"15min":26.128},"min":{"1min":12.429,"5min":11.631,"15min":8.0559999999999992},"max":{"1min":19.111000000000001,"5min":453.96199999999999,"15min":705.04499999999996},"last":17.596}]},{"osd":3,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":14.789999999999999,"5min":27.626999999999999,"15min":590.51199999999994},"min":{"1min":9.7219999999999995,"5min":7.6799999999999997,"15min":5.9740000000000002},"max":{"1min":23.942,"5min":448.65100000000001,"15min":30725.612000000001},"last":13.631},{"interface":"front","average":{"1min":12.497,"5min":20.047999999999998,"15min":582.06899999999996},"min":{"1min":7.7690000000000001,"5min":6.3840000000000003,"15min":6.3840000000000003},"max":{"1min":19.815000000000001,"5min":336.62599999999998,"15min":30407.659},"last":14.272}]},{"osd":5,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":16.190999999999999,"5min":30.579000000000001,"15min":32.869},"min":{"1min":14.1,"5min":7.1440000000000001,"15min":7.1440000000000001},"max":{"1min":18.738,"5min":450.81700000000001,"15min":450.81700000000001},"last":19.007000000000001},{"interface":"front","average":{"1min":10.430999999999999,"5min":19.959,"15min":22.803999999999998},"min":{"1min":6.2779999999999996,"5min":6.2779999999999996,"15min":6.2779999999999996},"max":{"1min":20.254999999999999,"5min":340.04000000000002,"15min":340.04000000000002},"last":11.579000000000001}]},{"osd":6,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":13.007,"5min":22.312000000000001,"15min":22.866},"min":{"1min":8.1120000000000001,"5min":7.1689999999999996,"15min":6.1520000000000001},"max":{"1min":22.010000000000002,"5min":339.47199999999998,"15min":696.952},"last":15.211},{"interface":"front","average":{"1min":14.724,"5min":23.917999999999999,"15min":23.209},"min":{"1min":9.0289999999999999,"5min":6.3449999999999998,"15min":6.3449999999999998},"max":{"1min":20.968,"5min":498.26999999999998,"15min":598.86900000000003},"last":19.809999999999999}]},{"osd":7,"last update":"Sat Jul 29 14:04:52 2023","interfaces":[{"interface":"back","average":{"1min":12.805,"5min":24.097000000000001,"15min":24.802},"min":{"1min":8.7430000000000003,"5min":8.3399999999999999,"15min":7.7930000000000001},"max":{"1min":18.638000000000002,"5min":340.61900000000003,"15min":699.31799999999998},"last":18.289000000000001},{"interface":"front","average":{"1min":17.483000000000001,"5min":31.318999999999999,"15min":33.040999999999997},"min":{"1min":13.861000000000001,"5min":11.067,"15min":11.067},"max":{"1min":19.817,"5min":455.339,"15min":703.68200000000002},"last":21.388999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509876,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82180,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94289660,"statfs":{"total":96636764160,"available":96552611840,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":12.891999999999999,"5min":16.984000000000002,"15min":62.423000000000002},"min":{"1min":6.9390000000000001,"5min":5.2350000000000003,"15min":5.2350000000000003},"max":{"1min":29.821000000000002,"5min":224.93899999999999,"15min":9502.5869999999995},"last":15.316000000000001},{"interface":"front","average":{"1min":15.925000000000001,"5min":19.867000000000001,"15min":65.921999999999997},"min":{"1min":8.2050000000000001,"5min":6.1580000000000004,"15min":6.1580000000000004},"max":{"1min":31.303999999999998,"5min":130.48400000000001,"15min":9511.3719999999994},"last":9.9489999999999998}]},{"osd":1,"last update":"Sat Jul 29 14:04:49 2023","interfaces":[{"interface":"back","average":{"1min":17.140999999999998,"5min":970.20899999999995,"15min":335.46300000000002},"min":{"1min":7.5839999999999996,"5min":7.3369999999999997,"15min":4.8529999999999998},"max":{"1min":73.679000000000002,"5min":26903.921999999999,"15min":26903.921999999999},"last":11.355},{"interface":"front","average":{"1min":12.878,"5min":963.24699999999996,"15min":332.12599999999998},"min":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.3330000000000002},"max":{"1min":69.432000000000002,"5min":26893.022000000001,"15min":26893.022000000001},"last":8.9100000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":14.891999999999999,"5min":21.295999999999999,"15min":19.027000000000001},"min":{"1min":10.173,"5min":7.8609999999999998,"15min":6.6299999999999999},"max":{"1min":26.302,"5min":191.73099999999999,"15min":191.73099999999999},"last":18.035},{"interface":"front","average":{"1min":11.848000000000001,"5min":16.003,"15min":15.422000000000001},"min":{"1min":7.4690000000000003,"5min":5.5499999999999998,"15min":5.5499999999999998},"max":{"1min":18,"5min":189.40199999999999,"15min":189.40199999999999},"last":9.6140000000000008}]},{"osd":3,"last update":"Sat Jul 29 14:05:46 2023","interfaces":[{"interface":"back","average":{"1min":11.444000000000001,"5min":15.523999999999999,"15min":521.29999999999995},"min":{"1min":7.2489999999999997,"5min":6.335,"15min":6.335},"max":{"1min":37.712000000000003,"5min":187.852,"15min":30710.619999999999},"last":14.010999999999999},{"interface":"front","average":{"1min":15.505000000000001,"5min":18.006,"15min":520.84199999999998},"min":{"1min":10.118,"5min":8.8770000000000007,"15min":7.6180000000000003},"max":{"1min":38.539999999999999,"5min":101.88200000000001,"15min":30591.776999999998},"last":16.777000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:04:56 2023","interfaces":[{"interface":"back","average":{"1min":16.030000000000001,"5min":808.61300000000006,"15min":1019.547},"min":{"1min":8.0950000000000006,"5min":6.718,"15min":6.1070000000000002},"max":{"1min":73.055999999999997,"5min":25620.438999999998,"15min":32899.582000000002},"last":19.931999999999999},{"interface":"front","average":{"1min":16.128,"5min":828.35799999999995,"15min":1009.853},"min":{"1min":10.254,"5min":6.2089999999999996,"15min":6.2089999999999996},"max":{"1min":40.786999999999999,"5min":25979.712,"15min":32485.278999999999},"last":12.157999999999999}]},{"osd":5,"last update":"Sat Jul 29 14:05:04 2023","interfaces":[{"interface":"back","average":{"1min":17.489999999999998,"5min":21.376999999999999,"15min":24.847000000000001},"min":{"1min":11.52,"5min":9.4949999999999992,"15min":8.9290000000000003},"max":{"1min":34.677,"5min":75.25,"15min":189.423},"last":13.355},{"interface":"front","average":{"1min":14.324,"5min":18.073,"15min":21.690000000000001},"min":{"1min":8.6969999999999992,"5min":7.3330000000000002,"15min":7.3330000000000002},"max":{"1min":24.77,"5min":80.085999999999999,"15min":188.518},"last":20.928999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:05:46 2023","interfaces":[{"interface":"back","average":{"1min":15.869,"5min":19.315000000000001,"15min":19.748999999999999},"min":{"1min":12.207000000000001,"5min":7.1059999999999999,"15min":7.1059999999999999},"max":{"1min":19.248999999999999,"5min":74.337000000000003,"15min":200.55099999999999},"last":19.044},{"interface":"front","average":{"1min":14.26,"5min":19.501999999999999,"15min":21.978000000000002},"min":{"1min":9.8089999999999993,"5min":8.0790000000000006,"15min":8.0790000000000006},"max":{"1min":19.190999999999999,"5min":89.125,"15min":222.65199999999999},"last":14.654}]}]},{"osd":5,"up_from":704,"seq":3023656976775,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77876,"kb_used_data":6636,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94293964,"statfs":{"total":96636764160,"available":96557019136,"internally_reserved":0,"allocated":6795264,"data_stored":3844748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:04:00 2023","interfaces":[{"interface":"back","average":{"1min":23.48,"5min":22.623000000000001,"15min":41.000999999999998},"min":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":4.742},"max":{"1min":150.63,"5min":262.56099999999998,"15min":299.87799999999999},"last":13.294},{"interface":"front","average":{"1min":24.837,"5min":22.727,"15min":52.545999999999999},"min":{"1min":5.8259999999999996,"5min":4.6539999999999999,"15min":4.6539999999999999},"max":{"1min":111.14100000000001,"5min":253.80000000000001,"15min":370.00999999999999},"last":8.3510000000000009}]},{"osd":1,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":13.867000000000001,"5min":25.277000000000001,"15min":33.274999999999999},"min":{"1min":6.5419999999999998,"5min":5.3780000000000001,"15min":5.3780000000000001},"max":{"1min":91.590000000000003,"5min":362.58800000000002,"15min":362.58800000000002},"last":12.656000000000001},{"interface":"front","average":{"1min":15.465999999999999,"5min":34.079000000000001,"15min":56.872999999999998},"min":{"1min":7.5650000000000004,"5min":6.8680000000000003,"15min":6.8680000000000003},"max":{"1min":85.760000000000005,"5min":356.69900000000001,"15min":363.50999999999999},"last":11.919}]},{"osd":2,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.152000000000001,"5min":26.193000000000001,"15min":51.703000000000003},"min":{"1min":8.1880000000000006,"5min":7.032,"15min":7.032},"max":{"1min":151.804,"5min":312.755,"15min":384.024},"last":15.550000000000001},{"interface":"front","average":{"1min":16.754000000000001,"5min":20.221,"15min":25.251000000000001},"min":{"1min":7.782,"5min":7.2759999999999998,"15min":7.2709999999999999},"max":{"1min":102.09099999999999,"5min":228.55600000000001,"15min":228.55600000000001},"last":9.6020000000000003}]},{"osd":3,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":20.324999999999999,"5min":28.802,"15min":55.957999999999998},"min":{"1min":10.734,"5min":7.4100000000000001,"15min":7.4100000000000001},"max":{"1min":131.36600000000001,"5min":312.12200000000001,"15min":381.23200000000003},"last":16.341000000000001},{"interface":"front","average":{"1min":18.568999999999999,"5min":28.875,"15min":66.894999999999996},"min":{"1min":7.944,"5min":7.944,"15min":7.944},"max":{"1min":118.456,"5min":265.92700000000002,"15min":382.779},"last":8.9109999999999996}]},{"osd":4,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":19.385999999999999,"5min":1108.5630000000001,"15min":412.19600000000003},"min":{"1min":10.5,"5min":7.343,"15min":7.3070000000000004},"max":{"1min":103.768,"5min":23572.741999999998,"15min":23572.741999999998},"last":17.928999999999998},{"interface":"front","average":{"1min":20.039000000000001,"5min":1113.885,"15min":428.73500000000001},"min":{"1min":8.7260000000000009,"5min":6.8929999999999998,"15min":6.8929999999999998},"max":{"1min":137.57400000000001,"5min":23610.142,"15min":23610.142},"last":10.183999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":17.074000000000002,"5min":25.704999999999998,"15min":51.972000000000001},"min":{"1min":9.7360000000000007,"5min":8.1709999999999994,"15min":8.1709999999999994},"max":{"1min":41.347000000000001,"5min":263.983,"15min":450.17500000000001},"last":10.763},{"interface":"front","average":{"1min":18.436,"5min":26.684999999999999,"15min":57.575000000000003},"min":{"1min":11.185,"5min":8.1969999999999992,"15min":8.1969999999999992},"max":{"1min":63.707999999999998,"5min":221.99199999999999,"15min":417.303},"last":17.109000000000002}]},{"osd":7,"last update":"Sat Jul 29 14:05:05 2023","interfaces":[{"interface":"back","average":{"1min":21.550999999999998,"5min":28.064,"15min":47.530999999999999},"min":{"1min":12.340999999999999,"5min":11.222,"15min":10.602},"max":{"1min":104.718,"5min":284.48000000000002,"15min":341.30500000000001},"last":18.585999999999999},{"interface":"front","average":{"1min":18.460999999999999,"5min":27.806000000000001,"15min":53.993000000000002},"min":{"1min":12.089,"5min":11.042999999999999,"15min":11.042999999999999},"max":{"1min":46.353999999999999,"5min":191.33099999999999,"15min":329.86500000000001},"last":11.348000000000001}]}]},{"osd":7,"up_from":650,"seq":2791728742803,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59676,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312164,"statfs":{"total":96636764160,"available":96575655936,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:45 2023","interfaces":[{"interface":"back","average":{"1min":11.94,"5min":13.635,"15min":23.015000000000001},"min":{"1min":5.5890000000000004,"5min":5.4539999999999997,"15min":5.4539999999999997},"max":{"1min":28.5,"5min":75.968000000000004,"15min":293.94499999999999},"last":8.9390000000000001},{"interface":"front","average":{"1min":14.968999999999999,"5min":18.658999999999999,"15min":28.620000000000001},"min":{"1min":6.8179999999999996,"5min":6.8179999999999996,"15min":6.8179999999999996},"max":{"1min":29.143999999999998,"5min":116.952,"15min":358.267},"last":7.827}]},{"osd":1,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":19.048999999999999,"5min":1108.855,"15min":389.67099999999999},"min":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":4.6660000000000004},"max":{"1min":184.25800000000001,"5min":27546.963,"15min":27546.963},"last":12.455},{"interface":"front","average":{"1min":22.969999999999999,"5min":1111.8620000000001,"15min":385.10300000000001},"min":{"1min":8.6110000000000007,"5min":5.5010000000000003,"15min":5.5010000000000003},"max":{"1min":150.75700000000001,"5min":27546.280999999999,"15min":27546.280999999999},"last":10.013999999999999}]},{"osd":2,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":13.449999999999999,"5min":17.512,"15min":25.91},"min":{"1min":9.0869999999999997,"5min":9.0869999999999997,"15min":7.9100000000000001},"max":{"1min":15.819000000000001,"5min":103.09,"15min":461.83800000000002},"last":14.742000000000001},{"interface":"front","average":{"1min":8.734,"5min":13.756,"15min":21.891999999999999},"min":{"1min":7.0039999999999996,"5min":6.6239999999999997,"15min":6.6239999999999997},"max":{"1min":13.706,"5min":181.97900000000001,"15min":385.40800000000002},"last":7.2060000000000004}]},{"osd":3,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":10.454000000000001,"5min":17.827999999999999,"15min":1298.6089999999999},"min":{"1min":7.8300000000000001,"5min":7.6660000000000004,"15min":7.6660000000000004},"max":{"1min":16.02,"5min":242.197,"15min":34550.838000000003},"last":11.24},{"interface":"front","average":{"1min":15.295,"5min":23.765999999999998,"15min":1309.0609999999999},"min":{"1min":12.576000000000001,"5min":8.1509999999999998,"15min":6.9550000000000001},"max":{"1min":18.369,"5min":243.06399999999999,"15min":34556.381000000001},"last":16.323}]},{"osd":4,"last update":"Sat Jul 29 14:04:51 2023","interfaces":[{"interface":"back","average":{"1min":20.402999999999999,"5min":1074.817,"15min":900.84500000000003},"min":{"1min":6.7220000000000004,"5min":6.3719999999999999,"15min":6.3719999999999999},"max":{"1min":103.931,"5min":25146.41,"15min":27891.68},"last":13.048},{"interface":"front","average":{"1min":29.045999999999999,"5min":1099.6300000000001,"15min":918.40200000000004},"min":{"1min":9.7360000000000007,"5min":8.1630000000000003,"15min":7.2699999999999996},"max":{"1min":189.47900000000001,"5min":25434.151999999998,"15min":27883.419999999998},"last":15.391}]},{"osd":5,"last update":"Sat Jul 29 14:05:08 2023","interfaces":[{"interface":"back","average":{"1min":17.529,"5min":28.314,"15min":72.040999999999997},"min":{"1min":14.367000000000001,"5min":11.215,"15min":10.683},"max":{"1min":22.012,"5min":230.16800000000001,"15min":1265.123},"last":17.106999999999999},{"interface":"front","average":{"1min":12.343999999999999,"5min":25.687999999999999,"15min":76.659000000000006},"min":{"1min":9.609,"5min":7.71,"15min":7.5540000000000003},"max":{"1min":21.030999999999999,"5min":255.23699999999999,"15min":1234.9300000000001},"last":10.641999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":15.662000000000001,"5min":20.847000000000001,"15min":24.681000000000001},"min":{"1min":11.005000000000001,"5min":9.9740000000000002,"15min":7.8689999999999998},"max":{"1min":49.978000000000002,"5min":95.558999999999997,"15min":243.66499999999999},"last":11.808},{"interface":"front","average":{"1min":21.350999999999999,"5min":25.521999999999998,"15min":30.802},"min":{"1min":16.145,"5min":12.111000000000001,"15min":10.542},"max":{"1min":79.692999999999998,"5min":148.334,"15min":454.68799999999999},"last":17.91}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-29T14:05:55.680 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.681 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.681 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.681 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.682 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.683 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.684 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.686 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.686 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.687 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.687 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.688 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.688 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-07-29T13:28:12.371049+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=28, tm_sec=12, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.689 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.689 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-07-29T13:28:12.371049+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=28, tm_sec=12, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.690 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.690 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-07-29T13:28:12.371049+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=28, tm_sec=12, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.691 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-07-29T13:28:12.371049+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=28, tm_sec=12, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.691 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.693 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-07-29T13:28:12.371049+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=28, tm_sec=12, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.693 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.693 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-07-29T13:28:12.371049+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=28, tm_sec=12, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.694 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.694 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.694 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-07-29T13:28:12.371049+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=28, tm_sec=12, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.696 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.696 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.697 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.697 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.697 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.697 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.697 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.698 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.698 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.698 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.698 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:05:55.698 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-07-29T14:05:57.034 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:52.369 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:57.037 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:52.372 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:57.056 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:38:52.389 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:57.084 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:52.369 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:57.087 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:52.372 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:57.090 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:52.426 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:57.092 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:52.427 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:57.101 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:38:52.386 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:05:57.106 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:38:52.441 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:12.123 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:07.457 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:12.124 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:07.459 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:12.151 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:07.481 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:12.181 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:07.456 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:12.181 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:07.458 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:12.185 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:07.519 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:12.186 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:07.521 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:12.195 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:07.474 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:12.201 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:07.535 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:12.984 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:08.269 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:12.985 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:08.270 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:13.006 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:08.278 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:13.011 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:08.344 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:13.011 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:08.345 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:13.019 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:08.354 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:13.045 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:08.377 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:13.046 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:08.378 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:13.052 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:08.387 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:15.700 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-29T14:06:15.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.821+0000 7f6b73124640 10 monclient: get_monmap_and_config 2023-07-29T14:06:15.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.821+0000 7f6b73124640 10 monclient: build_initial_monmap 2023-07-29T14:06:15.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.821+0000 7f6b73124640 10 monclient: monmap: 2023-07-29T14:06:15.843 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:06:15.843 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:15.843 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:06:15.823012+0000 2023-07-29T14:06:15.843 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:06:15.823012+0000 2023-07-29T14:06:15.843 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:06:15.843 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:15.852 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:06:15.852 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:06:15.852 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:06:15.852 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:06:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:06:15.853 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.822+0000 7f6b73124640 1 Processor -- start 2023-07-29T14:06:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.822+0000 7f6b73124640 1 -- start start 2023-07-29T14:06:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.822+0000 7f6b73124640 10 monclient: init 2023-07-29T14:06:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 10 monclient: _reopen_session rank -1 2023-07-29T14:06:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2023-07-29T14:06:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c0afb60 0x7f6b6c070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 10 monclient(hunting): picked mon.noname-b con 0x7f6b6c0afb60 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:06:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6b6c140f80 addr v1:172.21.15.40:6789/0 2023-07-29T14:06:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b6c13cd40 0x7f6b6c13d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 10 monclient(hunting): picked mon.noname-c con 0x7f6b6c13cd40 addr v2:172.21.15.45:3300/0 2023-07-29T14:06:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c0aff30 con 0x7f6b6c140f80 2023-07-29T14:06:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c0ab3b0 con 0x7f6b6c13cd40 2023-07-29T14:06:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0ab600 con 0x7f6b6c0afb60 2023-07-29T14:06:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 10 monclient(hunting): _renew_subs 2023-07-29T14:06:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b73124640 10 monclient(hunting): authenticate will time out at 3697.956364s 2023-07-29T14:06:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b70ec0640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c0afb60 0x7f6b6c070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.823+0000 7f6b716c1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b6c13cd40 0x7f6b6c13d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:15.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b70ec0640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c0afb60 0x7f6b6c070e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:46028/0 (socket says 172.21.15.40:46028) 2023-07-29T14:06:15.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b70ec0640 1 -- 172.21.15.40:0/744967095 learned_addr learned my addr 172.21.15.40:0/744967095 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:06:15.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b70ec0640 10 monclient(hunting): get_auth_request con 0x7f6b6c0afb60 auth_method 0 2023-07-29T14:06:15.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b70ec0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:15.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b70ec0640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:15.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b70ec0640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:15.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b716c1640 10 monclient(hunting): get_auth_request con 0x7f6b6c13cd40 auth_method 0 2023-07-29T14:06:15.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b716c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:15.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b716c1640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:15.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b716c1640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:15.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b716c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:15.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b716c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:15.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b716c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:15.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b70ec0640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:15.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b70ec0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:15.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.824+0000 7f6b70ec0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:15.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.826+0000 7f6b70ec0640 10 monclient(hunting): handle_auth_done global_id 16442 payload 274 2023-07-29T14:06:15.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.826+0000 7f6b70ec0640 1 -- 172.21.15.40:0/744967095 >> v2:172.21.15.45:3300/0 conn(0x7f6b6c13cd40 msgr2=0x7f6b6c13d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:06:15.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.826+0000 7f6b70ec0640 1 --2- 172.21.15.40:0/744967095 >> v2:172.21.15.45:3300/0 conn(0x7f6b6c13cd40 0x7f6b6c13d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:15.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.826+0000 7f6b70ec0640 1 -- 172.21.15.40:0/744967095 >> v1:172.21.15.40:6789/0 conn(0x7f6b6c140f80 legacy=0x7f6b6c1413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:06:15.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.826+0000 7f6b70ec0640 10 monclient: _finish_hunting 0 2023-07-29T14:06:15.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.826+0000 7f6b70ec0640 1 monclient: found mon.noname-b 2023-07-29T14:06:15.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.826+0000 7f6b70ec0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:06:15.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.826+0000 7f6b70ec0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:06:15.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.826+0000 7f6b70ec0640 1 -- 172.21.15.40:0/744967095 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0a9ec0 con 0x7f6b6c0afb60 2023-07-29T14:06:15.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.828+0000 7f6b70ec0640 1 --2- 172.21.15.40:0/744967095 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c0afb60 0x7f6b6c070e40 secure :-1 s=READY pgs=3242 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009870 tx=0x7f6b6002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd65426dd046f162 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:06:15.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 1 -- 172.21.15.40:0/744967095 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60031640 con 0x7f6b6c0afb60 2023-07-29T14:06:15.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:15.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:06:15.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 10 monclient: dump: 2023-07-29T14:06:15.888 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:15.888 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:15.888 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:15.888 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:15.888 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:15.888 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:15.889 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:15.889 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:15.889 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:15.889 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:15.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 10 monclient: _finish_auth 0 2023-07-29T14:06:15.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 10 monclient: _check_auth_tickets 2023-07-29T14:06:15.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:06:15.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 1 -- 172.21.15.40:0/744967095 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b600317a0 con 0x7f6b6c0afb60 2023-07-29T14:06:15.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b73124640 5 monclient: authenticate success, global_id 16442 2023-07-29T14:06:15.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b73124640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T14:06:15.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:06:15.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b73124640 10 monclient: get_monmap_and_config success 2023-07-29T14:06:15.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 1 -- 172.21.15.40:0/744967095 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60031b30 con 0x7f6b6c0afb60 2023-07-29T14:06:15.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:15.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b73124640 4 set_mon_vals no callback set 2023-07-29T14:06:15.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:06:15.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.832+0000 7f6b6b7fe640 10 monclient: dump: 2023-07-29T14:06:15.891 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:15.891 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:15.894 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:15.894 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:15.894 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:15.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.833+0000 7f6b73124640 10 monclient: shutdown 2023-07-29T14:06:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.833+0000 7f6b73124640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:06:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.833+0000 7f6b73124640 1 -- 172.21.15.40:0/744967095 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c0afb60 msgr2=0x7f6b6c070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.833+0000 7f6b73124640 1 --2- 172.21.15.40:0/744967095 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c0afb60 0x7f6b6c070e40 secure :-1 s=READY pgs=3242 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009870 tx=0x7f6b6002b8f0 comp rx=0 tx=0).stop 2023-07-29T14:06:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.833+0000 7f6b73124640 1 -- 172.21.15.40:0/744967095 shutdown_connections 2023-07-29T14:06:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.833+0000 7f6b73124640 1 --2- 172.21.15.40:0/744967095 >> v2:172.21.15.45:3300/0 conn(0x7f6b6c13cd40 0x7f6b6c13d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.833+0000 7f6b73124640 1 --2- 172.21.15.40:0/744967095 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c0afb60 0x7f6b6c070e40 unknown :-1 s=CLOSED pgs=3242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.833+0000 7f6b73124640 1 -- 172.21.15.40:0/744967095 >> 172.21.15.40:0/744967095 conn(0x7f6b6c065b00 msgr2=0x7f6b6c065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:06:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.833+0000 7f6b73124640 1 -- 172.21.15.40:0/744967095 shutdown_connections 2023-07-29T14:06:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.833+0000 7f6b73124640 1 -- 172.21.15.40:0/744967095 wait complete. 2023-07-29T14:06:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.833+0000 7f6b73124640 10 monclient: build_initial_monmap 2023-07-29T14:06:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.834+0000 7f6b73124640 10 monclient: monmap: 2023-07-29T14:06:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:06:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:06:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:06:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:06:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:06:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:06:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:06:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:06:15.905 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.834+0000 7f6b73124640 1 librados: starting msgr at 2023-07-29T14:06:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.834+0000 7f6b73124640 1 librados: starting objecter 2023-07-29T14:06:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.834+0000 7f6b73124640 1 Processor -- start 2023-07-29T14:06:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.834+0000 7f6b73124640 1 -- start start 2023-07-29T14:06:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.834+0000 7f6b73124640 1 librados: setting wanted keys 2023-07-29T14:06:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.834+0000 7f6b73124640 1 librados: calling monclient init 2023-07-29T14:06:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.834+0000 7f6b73124640 10 monclient: init 2023-07-29T14:06:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 10 monclient: _reopen_session rank -1 2023-07-29T14:06:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 10 monclient: _add_conns ranks=[0,2,1] 2023-07-29T14:06:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b6c0afb60 0x7f6b6c132d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 10 monclient(hunting): picked mon.noname-a con 0x7f6b6c0afb60 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:06:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c13cd40 0x7f6b6c0b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 10 monclient(hunting): picked mon.noname-c con 0x7f6b6c13cd40 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:06:15.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c140f80 0x7f6b6c0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:15.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 10 monclient(hunting): picked mon.noname-b con 0x7f6b6c140f80 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:06:15.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:15.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0b28c0 con 0x7f6b6c0afb60 2023-07-29T14:06:15.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c137d80 con 0x7f6b6c140f80 2023-07-29T14:06:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c138060 con 0x7f6b6c13cd40 2023-07-29T14:06:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 10 monclient(hunting): _renew_subs 2023-07-29T14:06:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b73124640 10 monclient(hunting): authenticate will time out at 3697.968377s 2023-07-29T14:06:15.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b6bfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c13cd40 0x7f6b6c0b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:15.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.835+0000 7f6b716c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c140f80 0x7f6b6c0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b6bfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c13cd40 0x7f6b6c0b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:46036/0 (socket says 172.21.15.40:46036) 2023-07-29T14:06:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b6bfff640 1 -- 172.21.15.40:0/566288037 learned_addr learned my addr 172.21.15.40:0/566288037 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:06:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b6bfff640 10 monclient(hunting): get_auth_request con 0x7f6b6c13cd40 auth_method 0 2023-07-29T14:06:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b6bfff640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b6bfff640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b716c1640 10 monclient(hunting): get_auth_request con 0x7f6b6c140f80 auth_method 0 2023-07-29T14:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b716c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b716c1640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b716c1640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b716c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b716c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.836+0000 7f6b716c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.839+0000 7f6b6bfff640 10 monclient(hunting): handle_auth_done global_id 16445 payload 824 2023-07-29T14:06:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.839+0000 7f6b6bfff640 1 -- 172.21.15.40:0/566288037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c140f80 msgr2=0x7f6b6c0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.839+0000 7f6b6bfff640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c140f80 0x7f6b6c0b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.839+0000 7f6b6bfff640 1 -- 172.21.15.40:0/566288037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b6c0afb60 msgr2=0x7f6b6c132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.839+0000 7f6b6bfff640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b6c0afb60 0x7f6b6c132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.839+0000 7f6b6bfff640 10 monclient: _finish_hunting 0 2023-07-29T14:06:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.839+0000 7f6b6bfff640 1 monclient: found mon.noname-c 2023-07-29T14:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.839+0000 7f6b6bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.839+0000 7f6b6bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.839+0000 7f6b6bfff640 1 -- 172.21.15.40:0/566288037 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600094d0 con 0x7f6b6c13cd40 2023-07-29T14:06:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.842+0000 7f6b6bfff640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c13cd40 0x7f6b6c0b22c0 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800b410 tx=0x7f6b5800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:06:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.842+0000 7f6b697fa640 10 client.?.objecter ms_handle_connect 0x7f6b6c13cd40 2023-07-29T14:06:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.842+0000 7f6b697fa640 10 client.?.objecter resend_mon_ops 2023-07-29T14:06:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 1 -- 172.21.15.40:0/566288037 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58011830 con 0x7f6b6c13cd40 2023-07-29T14:06:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:06:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 10 monclient: dump: 2023-07-29T14:06:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:15.931 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 10 monclient: _finish_auth 0 2023-07-29T14:06:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 10 monclient: _check_auth_tickets 2023-07-29T14:06:15.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:06:15.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 1 -- 172.21.15.40:0/566288037 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b58011990 con 0x7f6b6c13cd40 2023-07-29T14:06:15.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:06:15.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 1 -- 172.21.15.40:0/566288037 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58011d40 con 0x7f6b6c13cd40 2023-07-29T14:06:15.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:15.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:06:15.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b697fa640 10 monclient: dump: 2023-07-29T14:06:15.934 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:15.934 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:15.934 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:15.934 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:15.934 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:15.934 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:15.934 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:15.935 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:15.935 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:15.935 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:15.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b73124640 5 monclient: authenticate success, global_id 16445 2023-07-29T14:06:15.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b73124640 10 monclient: _renew_subs 2023-07-29T14:06:15.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b73124640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:06:15.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b73124640 1 -- 172.21.15.40:0/566288037 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c13d880 con 0x7f6b6c13cd40 2023-07-29T14:06:15.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b6affd640 4 set_mon_vals no callback set 2023-07-29T14:06:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b73124640 10 client.16445.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:06:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b73124640 10 monclient: _renew_subs 2023-07-29T14:06:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b73124640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:06:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b73124640 1 -- 172.21.15.40:0/566288037 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c0b3710 con 0x7f6b6c13cd40 2023-07-29T14:06:15.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.845+0000 7f6b73124640 1 librados: init done 2023-07-29T14:06:15.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.847+0000 7f6b73124640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:06:15.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.847+0000 7f6b73124640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:06:15.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.847+0000 7f6b73124640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:06:15.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.847+0000 7f6b73124640 1 -- 172.21.15.40:0/566288037 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6c04e000 con 0x7f6b6c13cd40 2023-07-29T14:06:15.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.867+0000 7f6b697fa640 1 -- 172.21.15.40:0/566288037 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f6b5801a400 con 0x7f6b6c13cd40 2023-07-29T14:06:15.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.868+0000 7f6b697fa640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f6b5006bf60 0x7f6b5006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:15.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.868+0000 7f6b70ec0640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f6b5006bf60 0x7f6b5006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.868+0000 7f6b70ec0640 10 monclient: get_auth_request con 0x7f6b5006bf60 auth_method 0 2023-07-29T14:06:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.868+0000 7f6b697fa640 10 client.16445.objecter ms_handle_connect 0x7f6b5006bf60 2023-07-29T14:06:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.868+0000 7f6b70ec0640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f6b5006bf60 0x7f6b5006e410 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6b60036000 tx=0x7f6b60030000 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:06:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.871+0000 7f6b697fa640 1 -- 172.21.15.40:0/566288037 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f6b5805c290 con 0x7f6b6c13cd40 2023-07-29T14:06:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.871+0000 7f6b697fa640 10 client.16445.objecter ms_dispatch 0x7f6b6c000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:06:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.871+0000 7f6b697fa640 3 client.16445.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:06:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.871+0000 7f6b697fa640 3 client.16445.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:06:15.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:15.871+0000 7f6b697fa640 20 client.16445.objecter dump_active .. 0 homeless 2023-07-29T14:06:16.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.173+0000 7f6b697fa640 1 -- 172.21.15.40:0/566288037 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f6b580532c0 con 0x7f6b6c13cd40 2023-07-29T14:06:16.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.173+0000 7f6b697fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:06:16.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.173+0000 7f6b697fa640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:06:16.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.356+0000 7f6b73124640 1 -- 172.21.15.40:0/566288037 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b6c0b2df0 con 0x7f6b5006bf60 2023-07-29T14:06:16.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.361+0000 7f6b697fa640 1 -- 172.21.15.40:0/566288037 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152356 (secure 0 0 0) 0x7f6b6c0b2df0 con 0x7f6b5006bf60 2023-07-29T14:06:16.363 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T14:06:16.364 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-29T14:06:16.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.363+0000 7f6b73124640 10 librados: watch_flush enter 2023-07-29T14:06:16.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.363+0000 7f6b73124640 10 librados: watch_flush exit 2023-07-29T14:06:16.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.363+0000 7f6b73124640 20 client.16445.objecter shutdown clearing up homeless session... 2023-07-29T14:06:16.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.363+0000 7f6b73124640 10 client.16445.objecter successfully canceled tick 2023-07-29T14:06:16.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 -- 172.21.15.40:0/566288037 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f6b5006bf60 msgr2=0x7f6b5006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:16.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f6b5006bf60 0x7f6b5006e410 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6b60036000 tx=0x7f6b60030000 comp rx=0 tx=0).stop 2023-07-29T14:06:16.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 10 monclient: shutdown 2023-07-29T14:06:16.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:06:16.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 -- 172.21.15.40:0/566288037 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c13cd40 msgr2=0x7f6b6c0b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:16.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c13cd40 0x7f6b6c0b22c0 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800b410 tx=0x7f6b5800b8d0 comp rx=0 tx=0).stop 2023-07-29T14:06:16.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 -- 172.21.15.40:0/566288037 shutdown_connections 2023-07-29T14:06:16.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f6b5006bf60 0x7f6b5006e410 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:16.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c140f80 0x7f6b6c0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:16.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b6c13cd40 0x7f6b6c0b22c0 unknown :-1 s=CLOSED pgs=3243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:16.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 --2- 172.21.15.40:0/566288037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b6c0afb60 0x7f6b6c132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:16.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 -- 172.21.15.40:0/566288037 >> 172.21.15.40:0/566288037 conn(0x7f6b6c065b00 msgr2=0x7f6b6c0ac170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:06:16.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 -- 172.21.15.40:0/566288037 shutdown_connections 2023-07-29T14:06:16.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 -- 172.21.15.40:0/566288037 wait complete. 2023-07-29T14:06:16.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:16.364+0000 7f6b73124640 1 librados: shutdown 2023-07-29T14:06:16.375 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":2149,"stamp":"2023-07-29T14:06:15.637113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":520568,"kb_used_data":50744,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754454152,"statfs":{"total":773094113280,"available":772561051648,"internally_reserved":0,"allocated":51961856,"data_stored":28370954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002917"},"pg_stats":[{"pgid":"7.1b","version":"0'0","reported_seq":981,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:00.811211+0000","last_change":"2023-07-29T14:06:00.817006+0000","last_active":"2023-07-29T14:06:00.811211+0000","last_peered":"2023-07-29T14:06:00.811211+0000","last_clean":"2023-07-29T14:06:00.811211+0000","last_became_active":"2023-07-29T13:56:55.656930+0000","last_became_peered":"2023-07-29T13:56:55.656930+0000","last_unstale":"2023-07-29T14:06:00.811211+0000","last_undegraded":"2023-07-29T14:06:00.811211+0000","last_fullsized":"2023-07-29T14:06:00.811211+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:00.810385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:00.810385+0000","last_clean_scrub_stamp":"2023-07-29T14:06:00.810385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:58:40.689965+0000","scrub_duration":0.070146252000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":990,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:07.186288+0000","last_change":"2023-07-29T14:06:07.190582+0000","last_active":"2023-07-29T14:06:07.186288+0000","last_peered":"2023-07-29T14:06:07.186288+0000","last_clean":"2023-07-29T14:06:07.186288+0000","last_became_active":"2023-07-29T13:41:38.322679+0000","last_became_peered":"2023-07-29T13:41:38.322679+0000","last_unstale":"2023-07-29T14:06:07.186288+0000","last_undegraded":"2023-07-29T14:06:07.186288+0000","last_fullsized":"2023-07-29T14:06:07.186288+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:07.185413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:07.185413+0000","last_clean_scrub_stamp":"2023-07-29T14:06:07.185413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:26:55.323205+0000","scrub_duration":0.040433750999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":978,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:06.240995+0000","last_change":"2023-07-29T14:06:06.245535+0000","last_active":"2023-07-29T14:06:06.240995+0000","last_peered":"2023-07-29T14:06:06.240995+0000","last_clean":"2023-07-29T14:06:06.240995+0000","last_became_active":"2023-07-29T13:29:52.099163+0000","last_became_peered":"2023-07-29T13:29:52.099163+0000","last_unstale":"2023-07-29T14:06:06.240995+0000","last_undegraded":"2023-07-29T14:06:06.240995+0000","last_fullsized":"2023-07-29T14:06:06.240995+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:06.240199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:06.240199+0000","last_clean_scrub_stamp":"2023-07-29T14:06:06.240199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:43:50.331024+0000","scrub_duration":0.050444346000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":944,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:54.861717+0000","last_change":"2023-07-29T14:05:54.863494+0000","last_active":"2023-07-29T14:05:54.861717+0000","last_peered":"2023-07-29T14:05:54.861717+0000","last_clean":"2023-07-29T14:05:54.861717+0000","last_became_active":"2023-07-29T13:29:41.782506+0000","last_became_peered":"2023-07-29T13:29:41.782506+0000","last_unstale":"2023-07-29T14:05:54.861717+0000","last_undegraded":"2023-07-29T14:05:54.861717+0000","last_fullsized":"2023-07-29T14:05:54.861717+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:54.860920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:54.860920+0000","last_clean_scrub_stamp":"2023-07-29T14:05:54.860920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:14:42.977652+0000","scrub_duration":0.035476684000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":960,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:49.198542+0000","last_change":"2023-07-29T14:05:49.205817+0000","last_active":"2023-07-29T14:05:49.198542+0000","last_peered":"2023-07-29T14:05:49.198542+0000","last_clean":"2023-07-29T14:05:49.198542+0000","last_became_active":"2023-07-29T14:05:38.060277+0000","last_became_peered":"2023-07-29T14:05:38.060277+0000","last_unstale":"2023-07-29T14:05:49.198542+0000","last_undegraded":"2023-07-29T14:05:49.198542+0000","last_fullsized":"2023-07-29T14:05:49.198542+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:49.197193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:49.197193+0000","last_clean_scrub_stamp":"2023-07-29T14:05:49.197193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:55:48.982266+0000","scrub_duration":0.090687935999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":927,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:14.022387+0000","last_change":"2023-07-29T14:06:14.024200+0000","last_active":"2023-07-29T14:06:14.022387+0000","last_peered":"2023-07-29T14:06:14.022387+0000","last_clean":"2023-07-29T14:06:14.022387+0000","last_became_active":"2023-07-29T13:53:19.600386+0000","last_became_peered":"2023-07-29T13:53:19.600386+0000","last_unstale":"2023-07-29T14:06:14.022387+0000","last_undegraded":"2023-07-29T14:06:14.022387+0000","last_fullsized":"2023-07-29T14:06:14.022387+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:14.021538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:14.021538+0000","last_clean_scrub_stamp":"2023-07-29T14:06:14.021538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:51:32.345412+0000","scrub_duration":0.060272523000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":949,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:04.307469+0000","last_change":"2023-07-29T14:06:04.309240+0000","last_active":"2023-07-29T14:06:04.307469+0000","last_peered":"2023-07-29T14:06:04.307469+0000","last_clean":"2023-07-29T14:06:04.307469+0000","last_became_active":"2023-07-29T13:56:55.400591+0000","last_became_peered":"2023-07-29T13:56:55.400591+0000","last_unstale":"2023-07-29T14:06:04.307469+0000","last_undegraded":"2023-07-29T14:06:04.307469+0000","last_fullsized":"2023-07-29T14:06:04.307469+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:04.306657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:04.306657+0000","last_clean_scrub_stamp":"2023-07-29T14:06:04.306657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:17:55.974071+0000","scrub_duration":0.081655448000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":967,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:53.019638+0000","last_change":"2023-07-29T14:05:53.021396+0000","last_active":"2023-07-29T14:05:53.019638+0000","last_peered":"2023-07-29T14:05:53.019638+0000","last_clean":"2023-07-29T14:05:53.019638+0000","last_became_active":"2023-07-29T13:29:31.072042+0000","last_became_peered":"2023-07-29T13:29:31.072042+0000","last_unstale":"2023-07-29T14:05:53.019638+0000","last_undegraded":"2023-07-29T14:05:53.019638+0000","last_fullsized":"2023-07-29T14:05:53.019638+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:53.018835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:53.018835+0000","last_clean_scrub_stamp":"2023-07-29T14:05:53.018835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:48:12.562623+0000","scrub_duration":0.032764568000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":915,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:49.563630+0000","last_change":"2023-07-29T13:53:19.677067+0000","last_active":"2023-07-29T14:05:49.563630+0000","last_peered":"2023-07-29T14:05:49.563630+0000","last_clean":"2023-07-29T14:05:49.563630+0000","last_became_active":"2023-07-29T13:53:19.671215+0000","last_became_peered":"2023-07-29T13:53:19.671215+0000","last_unstale":"2023-07-29T14:05:49.563630+0000","last_undegraded":"2023-07-29T14:05:49.563630+0000","last_fullsized":"2023-07-29T14:05:49.563630+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":999,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:55.184407+0000","last_change":"2023-07-29T14:05:55.189991+0000","last_active":"2023-07-29T14:05:55.184407+0000","last_peered":"2023-07-29T14:05:55.184407+0000","last_clean":"2023-07-29T14:05:55.184407+0000","last_became_active":"2023-07-29T13:56:55.511490+0000","last_became_peered":"2023-07-29T13:56:55.511490+0000","last_unstale":"2023-07-29T14:05:55.184407+0000","last_undegraded":"2023-07-29T14:05:55.184407+0000","last_fullsized":"2023-07-29T14:05:55.184407+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:55.183058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:55.183058+0000","last_clean_scrub_stamp":"2023-07-29T14:05:55.183058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:12:19.154130+0000","scrub_duration":0.037892422000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":981,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:54.998356+0000","last_change":"2023-07-29T14:05:55.000139+0000","last_active":"2023-07-29T14:05:54.998356+0000","last_peered":"2023-07-29T14:05:54.998356+0000","last_clean":"2023-07-29T14:05:54.998356+0000","last_became_active":"2023-07-29T14:05:37.617822+0000","last_became_peered":"2023-07-29T14:05:37.617822+0000","last_unstale":"2023-07-29T14:05:54.998356+0000","last_undegraded":"2023-07-29T14:05:54.998356+0000","last_fullsized":"2023-07-29T14:05:54.998356+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:54.997537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:54.997537+0000","last_clean_scrub_stamp":"2023-07-29T14:05:54.997537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:05:05.354390+0000","scrub_duration":0.060110108000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":926,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:12.963049+0000","last_change":"2023-07-29T14:06:12.964828+0000","last_active":"2023-07-29T14:06:12.963049+0000","last_peered":"2023-07-29T14:06:12.963049+0000","last_clean":"2023-07-29T14:06:12.963049+0000","last_became_active":"2023-07-29T13:53:19.637473+0000","last_became_peered":"2023-07-29T13:53:19.637473+0000","last_unstale":"2023-07-29T14:06:12.963049+0000","last_undegraded":"2023-07-29T14:06:12.963049+0000","last_fullsized":"2023-07-29T14:06:12.963049+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:12.962193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:12.962193+0000","last_clean_scrub_stamp":"2023-07-29T14:06:12.962193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:04:10.539578+0000","scrub_duration":0.040380061000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":934,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.800151+0000","last_change":"2023-07-29T13:29:33.530600+0000","last_active":"2023-07-29T14:05:39.800151+0000","last_peered":"2023-07-29T14:05:39.800151+0000","last_clean":"2023-07-29T14:05:39.800151+0000","last_became_active":"2023-07-29T13:29:33.515255+0000","last_became_peered":"2023-07-29T13:29:33.515255+0000","last_unstale":"2023-07-29T14:05:39.800151+0000","last_undegraded":"2023-07-29T14:05:39.800151+0000","last_fullsized":"2023-07-29T14:05:39.800151+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":998,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:00.296691+0000","last_change":"2023-07-29T14:06:00.298559+0000","last_active":"2023-07-29T14:06:00.296691+0000","last_peered":"2023-07-29T14:06:00.296691+0000","last_clean":"2023-07-29T14:06:00.296691+0000","last_became_active":"2023-07-29T14:05:37.510437+0000","last_became_peered":"2023-07-29T14:05:37.510437+0000","last_unstale":"2023-07-29T14:06:00.296691+0000","last_undegraded":"2023-07-29T14:06:00.296691+0000","last_fullsized":"2023-07-29T14:06:00.296691+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:00.295845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:00.295845+0000","last_clean_scrub_stamp":"2023-07-29T14:06:00.295845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:40:35.629879+0000","scrub_duration":0.036103258999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":940,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.358148+0000","last_change":"2023-07-29T13:56:55.146603+0000","last_active":"2023-07-29T14:05:38.358148+0000","last_peered":"2023-07-29T14:05:38.358148+0000","last_clean":"2023-07-29T14:05:38.358148+0000","last_became_active":"2023-07-29T13:56:55.139167+0000","last_became_peered":"2023-07-29T13:56:55.139167+0000","last_unstale":"2023-07-29T14:05:38.358148+0000","last_undegraded":"2023-07-29T14:05:38.358148+0000","last_fullsized":"2023-07-29T14:05:38.358148+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":964,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:52.809995+0000","last_change":"2023-07-29T14:05:52.811915+0000","last_active":"2023-07-29T14:05:52.809995+0000","last_peered":"2023-07-29T14:05:52.809995+0000","last_clean":"2023-07-29T14:05:52.809995+0000","last_became_active":"2023-07-29T14:05:37.614802+0000","last_became_peered":"2023-07-29T14:05:37.614802+0000","last_unstale":"2023-07-29T14:05:52.809995+0000","last_undegraded":"2023-07-29T14:05:52.809995+0000","last_fullsized":"2023-07-29T14:05:52.809995+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:52.809166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:52.809166+0000","last_clean_scrub_stamp":"2023-07-29T14:05:52.809166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:08:42.706414+0000","scrub_duration":0.036979765999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1002,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:54.148401+0000","last_change":"2023-07-29T14:05:54.154623+0000","last_active":"2023-07-29T14:05:54.148401+0000","last_peered":"2023-07-29T14:05:54.148401+0000","last_clean":"2023-07-29T14:05:54.148401+0000","last_became_active":"2023-07-29T13:56:55.065724+0000","last_became_peered":"2023-07-29T13:56:55.065724+0000","last_unstale":"2023-07-29T14:05:54.148401+0000","last_undegraded":"2023-07-29T14:05:54.148401+0000","last_fullsized":"2023-07-29T14:05:54.148401+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:54.147587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:54.147587+0000","last_clean_scrub_stamp":"2023-07-29T14:05:54.147587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:33:58.287861+0000","scrub_duration":0.044510354000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":985,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:56.293729+0000","last_change":"2023-07-29T14:05:56.293729+0000","last_active":"2023-07-29T14:05:56.293729+0000","last_peered":"2023-07-29T14:05:56.293729+0000","last_clean":"2023-07-29T14:05:56.293729+0000","last_became_active":"2023-07-29T13:28:14.423044+0000","last_became_peered":"2023-07-29T13:28:14.423044+0000","last_unstale":"2023-07-29T14:05:56.293729+0000","last_undegraded":"2023-07-29T14:05:56.293729+0000","last_fullsized":"2023-07-29T14:05:56.293729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:56.291155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:56.291155+0000","last_clean_scrub_stamp":"2023-07-29T14:05:56.291155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:42:54.557575+0000","scrub_duration":0.032224926000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1","version":"1004'8","reported_seq":997,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:52.065768+0000","last_change":"2023-07-29T14:05:52.069559+0000","last_active":"2023-07-29T14:05:52.065768+0000","last_peered":"2023-07-29T14:05:52.065768+0000","last_clean":"2023-07-29T14:05:52.065768+0000","last_became_active":"2023-07-29T13:41:38.521847+0000","last_became_peered":"2023-07-29T13:41:38.521847+0000","last_unstale":"2023-07-29T14:05:52.065768+0000","last_undegraded":"2023-07-29T14:05:52.065768+0000","last_fullsized":"2023-07-29T14:05:52.065768+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"1004'8","last_scrub_stamp":"2023-07-29T14:05:52.064254+0000","last_deep_scrub":"1004'8","last_deep_scrub_stamp":"2023-07-29T14:05:52.064254+0000","last_clean_scrub_stamp":"2023-07-29T14:05:52.064254+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T01:31:35.837707+0000","scrub_duration":0.082755203999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":957,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:57.236762+0000","last_change":"2023-07-29T14:05:57.242910+0000","last_active":"2023-07-29T14:05:57.236762+0000","last_peered":"2023-07-29T14:05:57.236762+0000","last_clean":"2023-07-29T14:05:57.236762+0000","last_became_active":"2023-07-29T13:56:55.258241+0000","last_became_peered":"2023-07-29T13:56:55.258241+0000","last_unstale":"2023-07-29T14:05:57.236762+0000","last_undegraded":"2023-07-29T14:05:57.236762+0000","last_fullsized":"2023-07-29T14:05:57.236762+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:57.235949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:57.235949+0000","last_clean_scrub_stamp":"2023-07-29T14:05:57.235949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:52:01.628478+0000","scrub_duration":0.034914097999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":969,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:51.505319+0000","last_change":"2023-07-29T13:29:52.230014+0000","last_active":"2023-07-29T14:05:51.505319+0000","last_peered":"2023-07-29T14:05:51.505319+0000","last_clean":"2023-07-29T14:05:51.505319+0000","last_became_active":"2023-07-29T13:29:52.224467+0000","last_became_peered":"2023-07-29T13:29:52.224467+0000","last_unstale":"2023-07-29T14:05:51.505319+0000","last_undegraded":"2023-07-29T14:05:51.505319+0000","last_fullsized":"2023-07-29T14:05:51.505319+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":986,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:03.192032+0000","last_change":"2023-07-29T14:06:03.193988+0000","last_active":"2023-07-29T14:06:03.192032+0000","last_peered":"2023-07-29T14:06:03.192032+0000","last_clean":"2023-07-29T14:06:03.192032+0000","last_became_active":"2023-07-29T13:28:14.642023+0000","last_became_peered":"2023-07-29T13:28:14.642023+0000","last_unstale":"2023-07-29T14:06:03.192032+0000","last_undegraded":"2023-07-29T14:06:03.192032+0000","last_fullsized":"2023-07-29T14:06:03.192032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:03.191182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:03.191182+0000","last_clean_scrub_stamp":"2023-07-29T14:06:03.191182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:04:17.112604+0000","scrub_duration":0.035278285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":946,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:08.699663+0000","last_change":"2023-07-29T14:06:08.701416+0000","last_active":"2023-07-29T14:06:08.699663+0000","last_peered":"2023-07-29T14:06:08.699663+0000","last_clean":"2023-07-29T14:06:08.699663+0000","last_became_active":"2023-07-29T13:29:41.844314+0000","last_became_peered":"2023-07-29T13:29:41.844314+0000","last_unstale":"2023-07-29T14:06:08.699663+0000","last_undegraded":"2023-07-29T14:06:08.699663+0000","last_fullsized":"2023-07-29T14:06:08.699663+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:08.698853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:08.698853+0000","last_clean_scrub_stamp":"2023-07-29T14:06:08.698853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:09:31.454798+0000","scrub_duration":0.035659037999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":977,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:02.941253+0000","last_change":"2023-07-29T14:06:02.941253+0000","last_active":"2023-07-29T14:06:02.941253+0000","last_peered":"2023-07-29T14:06:02.941253+0000","last_clean":"2023-07-29T14:06:02.941253+0000","last_became_active":"2023-07-29T13:56:55.150514+0000","last_became_peered":"2023-07-29T13:56:55.150514+0000","last_unstale":"2023-07-29T14:06:02.941253+0000","last_undegraded":"2023-07-29T14:06:02.941253+0000","last_fullsized":"2023-07-29T14:06:02.941253+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:02.938567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:02.938567+0000","last_clean_scrub_stamp":"2023-07-29T14:06:02.938567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:27:07.832610+0000","scrub_duration":0.032938768,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1007,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:39.654218+0000","last_change":"2023-07-29T13:56:55.615681+0000","last_active":"2023-07-29T14:05:39.654218+0000","last_peered":"2023-07-29T14:05:39.654218+0000","last_clean":"2023-07-29T14:05:39.654218+0000","last_became_active":"2023-07-29T13:56:55.609414+0000","last_became_peered":"2023-07-29T13:56:55.609414+0000","last_unstale":"2023-07-29T14:05:39.654218+0000","last_undegraded":"2023-07-29T14:05:39.654218+0000","last_fullsized":"2023-07-29T14:05:39.654218+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1025,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:52.312489+0000","last_change":"2023-07-29T14:05:52.318168+0000","last_active":"2023-07-29T14:05:52.312489+0000","last_peered":"2023-07-29T14:05:52.312489+0000","last_clean":"2023-07-29T14:05:52.312489+0000","last_became_active":"2023-07-29T13:56:55.543861+0000","last_became_peered":"2023-07-29T13:56:55.543861+0000","last_unstale":"2023-07-29T14:05:52.312489+0000","last_undegraded":"2023-07-29T14:05:52.312489+0000","last_fullsized":"2023-07-29T14:05:52.312489+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:52.310408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:52.310408+0000","last_clean_scrub_stamp":"2023-07-29T14:05:52.310408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:37:08.149976+0000","scrub_duration":0.039706038999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":963,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:05.233062+0000","last_change":"2023-07-29T14:06:05.234815+0000","last_active":"2023-07-29T14:06:05.233062+0000","last_peered":"2023-07-29T14:06:05.233062+0000","last_clean":"2023-07-29T14:06:05.233062+0000","last_became_active":"2023-07-29T14:05:37.607023+0000","last_became_peered":"2023-07-29T14:05:37.607023+0000","last_unstale":"2023-07-29T14:06:05.233062+0000","last_undegraded":"2023-07-29T14:06:05.233062+0000","last_fullsized":"2023-07-29T14:06:05.233062+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:05.232236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:05.232236+0000","last_clean_scrub_stamp":"2023-07-29T14:06:05.232236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:54:00.313000+0000","scrub_duration":0.036641874999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":986,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:53.331278+0000","last_change":"2023-07-29T14:05:53.333097+0000","last_active":"2023-07-29T14:05:53.331278+0000","last_peered":"2023-07-29T14:05:53.331278+0000","last_clean":"2023-07-29T14:05:53.331278+0000","last_became_active":"2023-07-29T13:28:14.708042+0000","last_became_peered":"2023-07-29T13:28:14.708042+0000","last_unstale":"2023-07-29T14:05:53.331278+0000","last_undegraded":"2023-07-29T14:05:53.331278+0000","last_fullsized":"2023-07-29T14:05:53.331278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:53.330469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:53.330469+0000","last_clean_scrub_stamp":"2023-07-29T14:05:53.330469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:53:28.809768+0000","scrub_duration":0.040276365000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"1004'78","reported_seq":1116,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:51.159032+0000","last_change":"2023-07-29T14:05:51.166709+0000","last_active":"2023-07-29T14:05:51.159032+0000","last_peered":"2023-07-29T14:05:51.159032+0000","last_clean":"2023-07-29T14:05:51.159032+0000","last_became_active":"2023-07-29T13:56:55.157702+0000","last_became_peered":"2023-07-29T13:56:55.157702+0000","last_unstale":"2023-07-29T14:05:51.159032+0000","last_undegraded":"2023-07-29T14:05:51.159032+0000","last_fullsized":"2023-07-29T14:05:51.159032+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"1004'78","last_scrub_stamp":"2023-07-29T14:05:51.155533+0000","last_deep_scrub":"1004'78","last_deep_scrub_stamp":"2023-07-29T14:05:51.155533+0000","last_clean_scrub_stamp":"2023-07-29T14:05:51.155533+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:17:55.983069+0000","scrub_duration":0.143700576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1008,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:50.973540+0000","last_change":"2023-07-29T14:05:50.975646+0000","last_active":"2023-07-29T14:05:50.973540+0000","last_peered":"2023-07-29T14:05:50.973540+0000","last_clean":"2023-07-29T14:05:50.973540+0000","last_became_active":"2023-07-29T13:56:55.341426+0000","last_became_peered":"2023-07-29T13:56:55.341426+0000","last_unstale":"2023-07-29T14:05:50.973540+0000","last_undegraded":"2023-07-29T14:05:50.973540+0000","last_fullsized":"2023-07-29T14:05:50.973540+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:50.972716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:50.972716+0000","last_clean_scrub_stamp":"2023-07-29T14:05:50.972716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:38:24.126097+0000","scrub_duration":0.034645589999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":959,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:01.261540+0000","last_change":"2023-07-29T14:06:01.263330+0000","last_active":"2023-07-29T14:06:01.261540+0000","last_peered":"2023-07-29T14:06:01.261540+0000","last_clean":"2023-07-29T14:06:01.261540+0000","last_became_active":"2023-07-29T13:41:38.057649+0000","last_became_peered":"2023-07-29T13:41:38.057649+0000","last_unstale":"2023-07-29T14:06:01.261540+0000","last_undegraded":"2023-07-29T14:06:01.261540+0000","last_fullsized":"2023-07-29T14:06:01.261540+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:01.260705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:01.260705+0000","last_clean_scrub_stamp":"2023-07-29T14:06:01.260705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:16:52.634948+0000","scrub_duration":0.065964967999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":938,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:03.967776+0000","last_change":"2023-07-29T14:06:03.973270+0000","last_active":"2023-07-29T14:06:03.967776+0000","last_peered":"2023-07-29T14:06:03.967776+0000","last_clean":"2023-07-29T14:06:03.967776+0000","last_became_active":"2023-07-29T13:53:19.560480+0000","last_became_peered":"2023-07-29T13:53:19.560480+0000","last_unstale":"2023-07-29T14:06:03.967776+0000","last_undegraded":"2023-07-29T14:06:03.967776+0000","last_fullsized":"2023-07-29T14:06:03.967776+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:03.966930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:03.966930+0000","last_clean_scrub_stamp":"2023-07-29T14:06:03.966930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:08:51.541717+0000","scrub_duration":0.044001921999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":926,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:10.990517+0000","last_change":"2023-07-29T14:06:10.996113+0000","last_active":"2023-07-29T14:06:10.990517+0000","last_peered":"2023-07-29T14:06:10.990517+0000","last_clean":"2023-07-29T14:06:10.990517+0000","last_became_active":"2023-07-29T13:53:19.611477+0000","last_became_peered":"2023-07-29T13:53:19.611477+0000","last_unstale":"2023-07-29T14:06:10.990517+0000","last_undegraded":"2023-07-29T14:06:10.990517+0000","last_fullsized":"2023-07-29T14:06:10.990517+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:10.989645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:10.989645+0000","last_clean_scrub_stamp":"2023-07-29T14:06:10.989645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:33:44.721397+0000","scrub_duration":0.077086057999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":952,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:38.348463+0000","last_change":"2023-07-29T13:56:55.182600+0000","last_active":"2023-07-29T14:05:38.348463+0000","last_peered":"2023-07-29T14:05:38.348463+0000","last_clean":"2023-07-29T14:05:38.348463+0000","last_became_active":"2023-07-29T13:56:55.173228+0000","last_became_peered":"2023-07-29T13:56:55.173228+0000","last_unstale":"2023-07-29T14:05:38.348463+0000","last_undegraded":"2023-07-29T14:05:38.348463+0000","last_fullsized":"2023-07-29T14:05:38.348463+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":940,"reported_epoch":1004,"state":"active+clean+scrubbing+deep","last_fresh":"2023-07-29T14:05:53.296254+0000","last_change":"2023-07-29T14:06:01.255819+0000","last_active":"2023-07-29T14:05:53.296254+0000","last_peered":"2023-07-29T14:05:53.296254+0000","last_clean":"2023-07-29T14:05:53.296254+0000","last_became_active":"2023-07-29T13:56:55.238036+0000","last_became_peered":"2023-07-29T13:56:55.238036+0000","last_unstale":"2023-07-29T14:05:53.296254+0000","last_undegraded":"2023-07-29T14:05:53.296254+0000","last_fullsized":"2023-07-29T14:05:53.296254+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T13:29:15.987740+0000","last_clean_scrub_stamp":"2023-07-29T13:29:15.987740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":944,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:11.796599+0000","last_change":"2023-07-29T14:06:11.802047+0000","last_active":"2023-07-29T14:06:11.796599+0000","last_peered":"2023-07-29T14:06:11.796599+0000","last_clean":"2023-07-29T14:06:11.796599+0000","last_became_active":"2023-07-29T13:29:33.508113+0000","last_became_peered":"2023-07-29T13:29:33.508113+0000","last_unstale":"2023-07-29T14:06:11.796599+0000","last_undegraded":"2023-07-29T14:06:11.796599+0000","last_fullsized":"2023-07-29T14:06:11.796599+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:11.795786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:11.795786+0000","last_clean_scrub_stamp":"2023-07-29T14:06:11.795786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:00:12.237098+0000","scrub_duration":0.046529005999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":953,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:55.179499+0000","last_change":"2023-07-29T14:05:55.185047+0000","last_active":"2023-07-29T14:05:55.179499+0000","last_peered":"2023-07-29T14:05:55.179499+0000","last_clean":"2023-07-29T14:05:55.179499+0000","last_became_active":"2023-07-29T13:29:32.123788+0000","last_became_peered":"2023-07-29T13:29:32.123788+0000","last_unstale":"2023-07-29T14:05:55.179499+0000","last_undegraded":"2023-07-29T14:05:55.179499+0000","last_fullsized":"2023-07-29T14:05:55.179499+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:55.178683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:55.178683+0000","last_clean_scrub_stamp":"2023-07-29T14:05:55.178683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:10:55.395109+0000","scrub_duration":0.070950845999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":949,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:11.147593+0000","last_change":"2023-07-29T14:06:11.152661+0000","last_active":"2023-07-29T14:06:11.147593+0000","last_peered":"2023-07-29T14:06:11.147593+0000","last_clean":"2023-07-29T14:06:11.147593+0000","last_became_active":"2023-07-29T13:56:55.591631+0000","last_became_peered":"2023-07-29T13:56:55.591631+0000","last_unstale":"2023-07-29T14:06:11.147593+0000","last_undegraded":"2023-07-29T14:06:11.147593+0000","last_fullsized":"2023-07-29T14:06:11.147593+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:11.146780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:11.146780+0000","last_clean_scrub_stamp":"2023-07-29T14:06:11.146780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:07:27.329639+0000","scrub_duration":0.031543092000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1016,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:10.187646+0000","last_change":"2023-07-29T14:06:10.190321+0000","last_active":"2023-07-29T14:06:10.187646+0000","last_peered":"2023-07-29T14:06:10.187646+0000","last_clean":"2023-07-29T14:06:10.187646+0000","last_became_active":"2023-07-29T13:56:55.564155+0000","last_became_peered":"2023-07-29T13:56:55.564155+0000","last_unstale":"2023-07-29T14:06:10.187646+0000","last_undegraded":"2023-07-29T14:06:10.187646+0000","last_fullsized":"2023-07-29T14:06:10.187646+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:10.186791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:10.186791+0000","last_clean_scrub_stamp":"2023-07-29T14:06:10.186791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T01:01:01.099504+0000","scrub_duration":0.066176493000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":986,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:00.245088+0000","last_change":"2023-07-29T14:06:00.247048+0000","last_active":"2023-07-29T14:06:00.245088+0000","last_peered":"2023-07-29T14:06:00.245088+0000","last_clean":"2023-07-29T14:06:00.245088+0000","last_became_active":"2023-07-29T13:53:19.433950+0000","last_became_peered":"2023-07-29T13:53:19.433950+0000","last_unstale":"2023-07-29T14:06:00.245088+0000","last_undegraded":"2023-07-29T14:06:00.245088+0000","last_fullsized":"2023-07-29T14:06:00.245088+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:00.244224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:00.244224+0000","last_clean_scrub_stamp":"2023-07-29T14:06:00.244224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:56:06.799170+0000","scrub_duration":0.044199228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":972,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:03.254844+0000","last_change":"2023-07-29T14:06:03.259347+0000","last_active":"2023-07-29T14:06:03.254844+0000","last_peered":"2023-07-29T14:06:03.254844+0000","last_clean":"2023-07-29T14:06:03.254844+0000","last_became_active":"2023-07-29T14:05:37.513918+0000","last_became_peered":"2023-07-29T14:05:37.513918+0000","last_unstale":"2023-07-29T14:06:03.254844+0000","last_undegraded":"2023-07-29T14:06:03.254844+0000","last_fullsized":"2023-07-29T14:06:03.254844+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:03.254014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:03.254014+0000","last_clean_scrub_stamp":"2023-07-29T14:06:03.254014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:25:43.226591+0000","scrub_duration":0.081281532000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":17,"seq":73014444479,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59436,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312404,"statfs":{"total":96636764160,"available":96575901696,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":15.343999999999999,"5min":16.634,"15min":170.43700000000001},"min":{"1min":6.3490000000000002,"5min":6.3330000000000002,"15min":4.7590000000000003},"max":{"1min":111.66,"5min":172.44399999999999,"15min":12751.485000000001},"last":33.856999999999999},{"interface":"front","average":{"1min":16.254000000000001,"5min":16.774000000000001,"15min":173.22},"min":{"1min":6.79,"5min":5.4500000000000002,"15min":5.4500000000000002},"max":{"1min":129.28399999999999,"5min":173.48699999999999,"15min":12747.76},"last":8.6449999999999996}]},{"osd":1,"last update":"Sat Jul 29 14:05:52 2023","interfaces":[{"interface":"back","average":{"1min":16.469999999999999,"5min":1542.5060000000001,"15min":525.53399999999999},"min":{"1min":6.0339999999999998,"5min":6.0339999999999998,"15min":6.0339999999999998},"max":{"1min":88.435000000000002,"5min":29038.089,"15min":29038.089},"last":31.75},{"interface":"front","average":{"1min":22.672999999999998,"5min":1548.559,"15min":529.87699999999995},"min":{"1min":6.0449999999999999,"5min":6.0449999999999999,"15min":6.0449999999999999},"max":{"1min":180.447,"5min":29039.032999999999,"15min":29039.032999999999},"last":30.969000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:52 2023","interfaces":[{"interface":"back","average":{"1min":29.186,"5min":23.222999999999999,"15min":21.166},"min":{"1min":7.1050000000000004,"5min":7.1050000000000004,"15min":7.1050000000000004},"max":{"1min":200.35400000000001,"5min":249.84800000000001,"15min":784.66200000000003},"last":9.5519999999999996},{"interface":"front","average":{"1min":27.800000000000001,"5min":22.724,"15min":19.367999999999999},"min":{"1min":7.6959999999999997,"5min":7.6959999999999997,"15min":7.6959999999999997},"max":{"1min":191.994,"5min":202.64500000000001,"15min":368.62200000000001},"last":30.109000000000002}]},{"osd":4,"last update":"Sat Jul 29 14:05:56 2023","interfaces":[{"interface":"back","average":{"1min":15.656000000000001,"5min":25.140999999999998,"15min":478.09899999999999},"min":{"1min":7.4889999999999999,"5min":6.4740000000000002,"15min":6.4740000000000002},"max":{"1min":63.453000000000003,"5min":312.73599999999999,"15min":28381.707999999999},"last":36.429000000000002},{"interface":"front","average":{"1min":23.655999999999999,"5min":28.308,"15min":473.64299999999997},"min":{"1min":10.977,"5min":9.3219999999999992,"15min":9.2739999999999991},"max":{"1min":137.84299999999999,"5min":242.63,"15min":28126.012999999999},"last":35.195}]},{"osd":5,"last update":"Sat Jul 29 14:06:06 2023","interfaces":[{"interface":"back","average":{"1min":35.478000000000002,"5min":22.574999999999999,"15min":32.390999999999998},"min":{"1min":10.343,"5min":8.4540000000000006,"15min":7.3689999999999998},"max":{"1min":216.00299999999999,"5min":216.00299999999999,"15min":488.72300000000001},"last":28.291},{"interface":"front","average":{"1min":35.033999999999999,"5min":21.722000000000001,"15min":22.565000000000001},"min":{"1min":10.356999999999999,"5min":8.1379999999999999,"15min":7.3840000000000003},"max":{"1min":212.72999999999999,"5min":212.72999999999999,"15min":212.72999999999999},"last":34.546999999999997}]},{"osd":6,"last update":"Sat Jul 29 14:05:52 2023","interfaces":[{"interface":"back","average":{"1min":25.501999999999999,"5min":20.158999999999999,"15min":22.632999999999999},"min":{"1min":9.1099999999999994,"5min":8.0129999999999999,"15min":7.7779999999999996},"max":{"1min":203.011,"5min":203.011,"15min":787.18499999999995},"last":37.192},{"interface":"front","average":{"1min":27.638999999999999,"5min":21.126000000000001,"15min":22.797000000000001},"min":{"1min":9.6300000000000008,"5min":8.5790000000000006,"15min":8.3170000000000002},"max":{"1min":205.321,"5min":205.321,"15min":646.87599999999998},"last":35.814999999999998}]},{"osd":7,"last update":"Sat Jul 29 14:05:52 2023","interfaces":[{"interface":"back","average":{"1min":32.484999999999999,"5min":23.815000000000001,"15min":44.118000000000002},"min":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"max":{"1min":208.702,"5min":208.702,"15min":648.54200000000003},"last":32.439999999999998},{"interface":"front","average":{"1min":24.163,"5min":20.629000000000001,"15min":34.918999999999997},"min":{"1min":8.6259999999999994,"5min":8.6259999999999994,"15min":8.6259999999999994},"max":{"1min":108.06399999999999,"5min":108.06399999999999,"15min":364.86500000000001},"last":37.859999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444491,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63536,"kb_used_data":6248,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94308304,"statfs":{"total":96636764160,"available":96571703296,"internally_reserved":0,"allocated":6397952,"data_stored":3446913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":13.247999999999999,"15min":130.82599999999999},"min":{"1min":5.3789999999999996,"5min":5.0209999999999999,"15min":4.476},"max":{"1min":29.634,"5min":93.590999999999994,"15min":10009.493},"last":11.375999999999999},{"interface":"front","average":{"1min":13.936999999999999,"5min":13.526,"15min":131.76599999999999},"min":{"1min":7.1299999999999999,"5min":6.2069999999999999,"15min":5.617},"max":{"1min":37.527999999999999,"5min":99.156999999999996,"15min":9995.5869999999995},"last":7.5039999999999996}]},{"osd":1,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":16.233000000000001,"5min":1412.377,"15min":481.59100000000001},"min":{"1min":6.7910000000000004,"5min":6.0899999999999999,"15min":6.0899999999999999},"max":{"1min":101.836,"5min":29343.018,"15min":29343.018},"last":9.5299999999999994},{"interface":"front","average":{"1min":16.434000000000001,"5min":1411.5540000000001,"15min":479.24299999999999},"min":{"1min":5.7990000000000004,"5min":5.7990000000000004,"15min":5.7990000000000004},"max":{"1min":137.94,"5min":29342.263999999999,"15min":29342.263999999999},"last":8.5790000000000006}]},{"osd":3,"last update":"Sat Jul 29 14:05:44 2023","interfaces":[{"interface":"back","average":{"1min":13.795,"5min":19.425999999999998,"15min":365.00299999999999},"min":{"1min":8.1449999999999996,"5min":6.7809999999999997,"15min":6.7809999999999997},"max":{"1min":30.794,"5min":331.92399999999998,"15min":29770.313999999998},"last":14.581},{"interface":"front","average":{"1min":13.835000000000001,"5min":21.202000000000002,"15min":358.64400000000001},"min":{"1min":7.351,"5min":7.351,"15min":6.4749999999999996},"max":{"1min":34.841000000000001,"5min":332.49900000000002,"15min":29586.387999999999},"last":15.945}]},{"osd":4,"last update":"Sat Jul 29 14:05:53 2023","interfaces":[{"interface":"back","average":{"1min":14.763,"5min":814.56100000000004,"15min":870.06399999999996},"min":{"1min":6.5810000000000004,"5min":6.5810000000000004,"15min":6.5810000000000004},"max":{"1min":80.256,"5min":25685.629000000001,"15min":32071.659},"last":10.763999999999999},{"interface":"front","average":{"1min":15.894,"5min":810.43799999999999,"15min":871.18799999999999},"min":{"1min":8.0410000000000004,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":52.790999999999997,"5min":25610.863000000001,"15min":32126.150000000001},"last":19.074000000000002}]},{"osd":5,"last update":"Sat Jul 29 14:06:11 2023","interfaces":[{"interface":"back","average":{"1min":15.445,"5min":18.456,"15min":35.558999999999997},"min":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":35.451000000000001,"5min":193.45400000000001,"15min":428.04000000000002},"last":15.226000000000001},{"interface":"front","average":{"1min":18.681000000000001,"5min":19.016999999999999,"15min":38.996000000000002},"min":{"1min":7.2960000000000003,"5min":6.891,"15min":6.891},"max":{"1min":77.212999999999994,"5min":153.148,"15min":430.15499999999997},"last":12.048999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:05:40 2023","interfaces":[{"interface":"back","average":{"1min":17.140000000000001,"5min":21.350999999999999,"15min":20.024000000000001},"min":{"1min":8.3450000000000006,"5min":8.3450000000000006,"15min":8.3450000000000006},"max":{"1min":77.855000000000004,"5min":318.45299999999997,"15min":318.45299999999997},"last":16.716000000000001},{"interface":"front","average":{"1min":17.119,"5min":21.058,"15min":20.518999999999998},"min":{"1min":9.2230000000000008,"5min":9.2230000000000008,"15min":7.5300000000000002},"max":{"1min":44.755000000000003,"5min":349.34899999999999,"15min":349.34899999999999},"last":12.656000000000001}]},{"osd":7,"last update":"Sat Jul 29 14:05:44 2023","interfaces":[{"interface":"back","average":{"1min":19.048999999999999,"5min":19.532,"15min":21.023},"min":{"1min":10.064,"5min":8.5079999999999991,"15min":7.5890000000000004},"max":{"1min":80.870000000000005,"5min":291.80900000000003,"15min":291.80900000000003},"last":13.271000000000001},{"interface":"front","average":{"1min":22.766999999999999,"5min":21.436,"15min":22.666},"min":{"1min":9.3379999999999992,"5min":9.1940000000000008,"15min":9.1940000000000008},"max":{"1min":82.373999999999995,"5min":291.44200000000001,"15min":291.44200000000001},"last":13.875}]}]},{"osd":1,"up_from":447,"seq":1919850381749,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59832,"kb_used_data":6640,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312008,"statfs":{"total":96636764160,"available":96575496192,"internally_reserved":0,"allocated":6799360,"data_stored":3844753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":20.931000000000001,"5min":20.015999999999998,"15min":112.807},"min":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.3680000000000003},"max":{"1min":116.512,"5min":364.47800000000001,"15min":12525.57},"last":15.936999999999999},{"interface":"front","average":{"1min":18.943000000000001,"5min":20.664000000000001,"15min":112.874},"min":{"1min":6.0970000000000004,"5min":6.0970000000000004,"15min":4.6820000000000004},"max":{"1min":100.402,"5min":372.00999999999999,"15min":12523.974},"last":13.132999999999999}]},{"osd":2,"last update":"Sat Jul 29 14:05:55 2023","interfaces":[{"interface":"back","average":{"1min":12.737,"5min":24.844999999999999,"15min":19.154},"min":{"1min":5.6580000000000004,"5min":5.6580000000000004,"15min":5.5919999999999996},"max":{"1min":40.566000000000003,"5min":360.88099999999997,"15min":360.88099999999997},"last":16.925000000000001},{"interface":"front","average":{"1min":13.837999999999999,"5min":25.808,"15min":20.196000000000002},"min":{"1min":6.6120000000000001,"5min":6.6120000000000001,"15min":6.2729999999999997},"max":{"1min":59.558999999999997,"5min":368.51400000000001,"15min":368.51400000000001},"last":13.722}]},{"osd":3,"last update":"Sat Jul 29 14:05:55 2023","interfaces":[{"interface":"back","average":{"1min":12.858000000000001,"5min":27.369,"15min":531.38300000000004},"min":{"1min":8.468,"5min":6.7140000000000004,"15min":6.7140000000000004},"max":{"1min":21.891999999999999,"5min":435.92200000000003,"15min":34504.942000000003},"last":17.591000000000001},{"interface":"front","average":{"1min":18.202999999999999,"5min":38.177999999999997,"15min":540.05899999999997},"min":{"1min":10.888,"5min":10.423,"15min":5.3639999999999999},"max":{"1min":37.070999999999998,"5min":700.10500000000002,"15min":34620.970999999998},"last":18.771000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:05:55 2023","interfaces":[{"interface":"back","average":{"1min":12.792,"5min":1168.114,"15min":980.44200000000001},"min":{"1min":7.9939999999999998,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":19.539999999999999,"5min":25247.098999999998,"15min":32258.529999999999},"last":9.798},{"interface":"front","average":{"1min":15.993,"5min":1162.9349999999999,"15min":967.01300000000003},"min":{"1min":9.3900000000000006,"5min":9.3900000000000006,"15min":6.5090000000000003},"max":{"1min":35.779000000000003,"5min":25174.008999999998,"15min":31933.992999999999},"last":19.774999999999999}]},{"osd":5,"last update":"Sat Jul 29 14:05:13 2023","interfaces":[{"interface":"back","average":{"1min":15.249000000000001,"5min":33.526000000000003,"15min":43.616},"min":{"1min":10.416,"5min":10.294,"15min":10.294},"max":{"1min":23.649999999999999,"5min":433.79300000000001,"15min":433.79300000000001},"last":25.573},{"interface":"front","average":{"1min":15.952,"5min":36.515000000000001,"15min":46.286000000000001},"min":{"1min":11.467000000000001,"5min":9.6929999999999996,"15min":8.7260000000000009},"max":{"1min":22.300999999999998,"5min":419.22800000000001,"15min":419.22800000000001},"last":27.488}]},{"osd":6,"last update":"Sat Jul 29 14:05:55 2023","interfaces":[{"interface":"back","average":{"1min":10.282999999999999,"5min":20.245999999999999,"15min":18.065999999999999},"min":{"1min":6.8520000000000003,"5min":6.6079999999999997,"15min":6.327},"max":{"1min":17.503,"5min":357.20499999999998,"15min":357.20499999999998},"last":8.0879999999999992},{"interface":"front","average":{"1min":13.545,"5min":25.114999999999998,"15min":22.295000000000002},"min":{"1min":7.9039999999999999,"5min":7.9039999999999999,"15min":7.9039999999999999},"max":{"1min":19.331,"5min":359.68099999999998,"15min":359.68099999999998},"last":11.286}]},{"osd":7,"last update":"Sat Jul 29 14:05:55 2023","interfaces":[{"interface":"back","average":{"1min":16.536999999999999,"5min":33.811,"15min":23.893999999999998},"min":{"1min":11.493,"5min":8.5199999999999996,"15min":7.5919999999999996},"max":{"1min":24.332999999999998,"5min":563.29700000000003,"15min":563.29700000000003},"last":14.318},{"interface":"front","average":{"1min":15.179,"5min":29.501000000000001,"15min":21.321999999999999},"min":{"1min":9.4250000000000007,"5min":8.2240000000000002,"15min":7.2919999999999998},"max":{"1min":35.567999999999998,"5min":352.65199999999999,"15min":352.65199999999999},"last":10.571}]}]},{"osd":0,"up_from":1002,"seq":4303557231035,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59436,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312404,"statfs":{"total":96636764160,"available":96575901696,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":71.099999999999994,"5min":1023.201,"15min":351.10899999999998},"min":{"1min":8.9689999999999994,"5min":5.9249999999999998,"15min":4.6269999999999998},"max":{"1min":350.00200000000001,"5min":27534,"15min":27534},"last":11.016},{"interface":"front","average":{"1min":99.875,"5min":1025.7080000000001,"15min":350.98899999999998},"min":{"1min":5.9589999999999996,"5min":5.7240000000000002,"15min":5.0739999999999998},"max":{"1min":591.69600000000003,"5min":27535.008999999998,"15min":27535.008999999998},"last":13.228}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.170999999999999,"5min":22.285,"15min":18.638000000000002},"min":{"1min":11.680999999999999,"5min":8.1069999999999993,"15min":6.6159999999999997},"max":{"1min":364.62799999999999,"5min":364.62799999999999,"15min":364.62799999999999},"last":12.390000000000001},{"interface":"front","average":{"1min":45.890000000000001,"5min":26.417999999999999,"15min":21.443000000000001},"min":{"1min":14.172000000000001,"5min":10.023999999999999,"15min":8.8780000000000001},"max":{"1min":370.34300000000002,"5min":370.34300000000002,"15min":370.34300000000002},"last":13.864000000000001}]},{"osd":3,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":66.832999999999998,"5min":25.178000000000001,"15min":355.89100000000002},"min":{"1min":6.7610000000000001,"5min":6.4409999999999998,"15min":5.891},"max":{"1min":444.59500000000003,"5min":444.59500000000003,"15min":30170.040000000001},"last":15.644},{"interface":"front","average":{"1min":65.754000000000005,"5min":33.606000000000002,"15min":363.07400000000001},"min":{"1min":10.266999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"max":{"1min":386.06099999999998,"5min":386.06099999999998,"15min":30171.194},"last":20.829999999999998}]},{"osd":4,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":92.555000000000007,"5min":36.014000000000003,"15min":585.43899999999996},"min":{"1min":12.201000000000001,"5min":6.4480000000000004,"15min":6.0979999999999999},"max":{"1min":445.65300000000002,"5min":445.65300000000002,"15min":29892.669999999998},"last":16.465},{"interface":"front","average":{"1min":115.806,"5min":40.560000000000002,"15min":588.88900000000001},"min":{"1min":7.923,"5min":5.9400000000000004,"15min":5.9400000000000004},"max":{"1min":593.13499999999999,"5min":593.13499999999999,"15min":29877.289000000001},"last":18.184000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.341000000000001,"5min":28.972000000000001,"15min":56.932000000000002},"min":{"1min":12.702,"5min":9.5700000000000003,"15min":9.5700000000000003},"max":{"1min":403.13999999999999,"5min":403.13999999999999,"15min":508.02100000000002},"last":14.475},{"interface":"front","average":{"1min":43.966999999999999,"5min":23.216000000000001,"15min":32.374000000000002},"min":{"1min":7.6429999999999998,"5min":7.1210000000000004,"15min":7.1210000000000004},"max":{"1min":590.58500000000004,"5min":590.58500000000004,"15min":590.58500000000004},"last":19.638000000000002}]},{"osd":6,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":52.365000000000002,"5min":30.853999999999999,"15min":25.227},"min":{"1min":16.876000000000001,"5min":10.409000000000001,"15min":9.9350000000000005},"max":{"1min":375.36500000000001,"5min":375.36500000000001,"15min":375.36500000000001},"last":17.614999999999998},{"interface":"front","average":{"1min":55.634,"5min":32.579999999999998,"15min":27.417999999999999},"min":{"1min":18.984999999999999,"5min":11.564,"15min":11.564},"max":{"1min":380.31599999999997,"5min":380.31599999999997,"15min":380.31599999999997},"last":17.045999999999999}]},{"osd":7,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":57.960999999999999,"5min":28.638000000000002,"15min":20.780000000000001},"min":{"1min":17.748999999999999,"5min":8.4939999999999998,"15min":7.9349999999999996},"max":{"1min":385.01900000000001,"5min":385.01900000000001,"15min":385.01900000000001},"last":22.012},{"interface":"front","average":{"1min":54.853999999999999,"5min":29.765999999999998,"15min":25.545000000000002},"min":{"1min":10.992000000000001,"5min":8.1229999999999993,"15min":8.1229999999999993},"max":{"1min":446.69099999999997,"5min":446.69099999999997,"15min":446.69099999999997},"last":20.251999999999999}]}]},{"osd":4,"up_from":16,"seq":68719477168,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59436,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312404,"statfs":{"total":96636764160,"available":96575901696,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":13.432,"5min":21.457000000000001,"15min":87.253},"min":{"1min":6.4029999999999996,"5min":4.6230000000000002,"15min":4.6230000000000002},"max":{"1min":48.048999999999999,"5min":495.86900000000003,"15min":10627.831},"last":9.6899999999999995},{"interface":"front","average":{"1min":18.010000000000002,"5min":25.013000000000002,"15min":90.852000000000004},"min":{"1min":7.2400000000000002,"5min":6.0990000000000002,"15min":5.6660000000000004},"max":{"1min":124.64100000000001,"5min":446.84100000000001,"15min":10653.877},"last":6.7069999999999999}]},{"osd":1,"last update":"Sat Jul 29 14:05:54 2023","interfaces":[{"interface":"back","average":{"1min":11.929,"5min":1441.175,"15min":495.887},"min":{"1min":7.21,"5min":7.21,"15min":6.7960000000000003},"max":{"1min":15.911,"5min":26872.857,"15min":26872.857},"last":5.9770000000000003},{"interface":"front","average":{"1min":12.18,"5min":1440.078,"15min":493.22500000000002},"min":{"1min":7.5720000000000001,"5min":7.4000000000000004,"15min":5.6909999999999998},"max":{"1min":19.245999999999999,"5min":26874.048999999999,"15min":26874.048999999999},"last":7.5110000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":12.445,"5min":24.050999999999998,"15min":23.050000000000001},"min":{"1min":8.9000000000000004,"5min":7.3019999999999996,"15min":7.3019999999999996},"max":{"1min":15.904999999999999,"5min":453.40100000000001,"15min":597.11800000000005},"last":15.567},{"interface":"front","average":{"1min":15.029999999999999,"5min":27.077999999999999,"15min":26.128},"min":{"1min":12.429,"5min":11.631,"15min":8.0559999999999992},"max":{"1min":19.111000000000001,"5min":453.96199999999999,"15min":705.04499999999996},"last":16.163}]},{"osd":3,"last update":"Sat Jul 29 14:05:54 2023","interfaces":[{"interface":"back","average":{"1min":16.591000000000001,"5min":27.045999999999999,"15min":590.73400000000004},"min":{"1min":13.207000000000001,"5min":9.2539999999999996,"15min":5.9740000000000002},"max":{"1min":43.826000000000001,"5min":448.65100000000001,"15min":30725.612000000001},"last":17.195},{"interface":"front","average":{"1min":14.129,"5min":20.515999999999998,"15min":580.85900000000004},"min":{"1min":6.641,"5min":6.3840000000000003,"15min":6.3840000000000003},"max":{"1min":45.087000000000003,"5min":336.62599999999998,"15min":30407.659},"last":10.534000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:06:06 2023","interfaces":[{"interface":"back","average":{"1min":19.056000000000001,"5min":28.420000000000002,"15min":31.863},"min":{"1min":11.228999999999999,"5min":7.1440000000000001,"15min":7.1440000000000001},"max":{"1min":58.966999999999999,"5min":450.81700000000001,"15min":450.81700000000001},"last":18.079000000000001},{"interface":"front","average":{"1min":13.9,"5min":19.434999999999999,"15min":22.106000000000002},"min":{"1min":7.3419999999999996,"5min":6.2779999999999996,"15min":6.2779999999999996},"max":{"1min":63.125,"5min":340.04000000000002,"15min":340.04000000000002},"last":11.733000000000001}]},{"osd":6,"last update":"Sat Jul 29 14:05:28 2023","interfaces":[{"interface":"back","average":{"1min":13.007,"5min":22.312000000000001,"15min":22.866},"min":{"1min":8.1120000000000001,"5min":7.1689999999999996,"15min":6.1520000000000001},"max":{"1min":22.010000000000002,"5min":339.47199999999998,"15min":696.952},"last":12.987},{"interface":"front","average":{"1min":14.724,"5min":23.917999999999999,"15min":23.209},"min":{"1min":9.0289999999999999,"5min":6.3449999999999998,"15min":6.3449999999999998},"max":{"1min":20.968,"5min":498.26999999999998,"15min":598.86900000000003},"last":13.667}]},{"osd":7,"last update":"Sat Jul 29 14:05:54 2023","interfaces":[{"interface":"back","average":{"1min":15.851000000000001,"5min":21.326000000000001,"15min":24.811},"min":{"1min":7.7300000000000004,"5min":7.7300000000000004,"15min":7.7300000000000004},"max":{"1min":20.425999999999998,"5min":340.61900000000003,"15min":699.31799999999998},"last":18.849},{"interface":"front","average":{"1min":15.250999999999999,"5min":28.420999999999999,"15min":32.200000000000003},"min":{"1min":8.7230000000000008,"5min":8.7230000000000008,"15min":8.7230000000000008},"max":{"1min":21.388999999999999,"5min":455.339,"15min":703.68200000000002},"last":12.41}]}]},{"osd":6,"up_from":15,"seq":64424509880,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81580,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94290260,"statfs":{"total":96636764160,"available":96553226240,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":12.891999999999999,"5min":16.984000000000002,"15min":62.423000000000002},"min":{"1min":6.9390000000000001,"5min":5.2350000000000003,"15min":5.2350000000000003},"max":{"1min":29.821000000000002,"5min":224.93899999999999,"15min":9502.5869999999995},"last":8.5259999999999998},{"interface":"front","average":{"1min":15.925000000000001,"5min":19.867000000000001,"15min":65.921999999999997},"min":{"1min":8.2050000000000001,"5min":6.1580000000000004,"15min":6.1580000000000004},"max":{"1min":31.303999999999998,"5min":130.48400000000001,"15min":9511.3719999999994},"last":13.576000000000001}]},{"osd":1,"last update":"Sat Jul 29 14:05:51 2023","interfaces":[{"interface":"back","average":{"1min":11.551,"5min":969.16300000000001,"15min":335.10300000000001},"min":{"1min":7.1070000000000002,"5min":7.1070000000000002,"15min":4.8529999999999998},"max":{"1min":15.645,"5min":26903.921999999999,"15min":26903.921999999999},"last":6.593},{"interface":"front","average":{"1min":10.327999999999999,"5min":963.16499999999996,"15min":331.75799999999998},"min":{"1min":5.3620000000000001,"5min":5.3620000000000001,"15min":5.3330000000000002},"max":{"1min":17.975999999999999,"5min":26893.022000000001,"15min":26893.022000000001},"last":12.176}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":14.891999999999999,"5min":21.295999999999999,"15min":19.027000000000001},"min":{"1min":10.173,"5min":7.8609999999999998,"15min":6.6299999999999999},"max":{"1min":26.302,"5min":191.73099999999999,"15min":191.73099999999999},"last":14.712},{"interface":"front","average":{"1min":11.848000000000001,"5min":16.003,"15min":15.422000000000001},"min":{"1min":7.4690000000000003,"5min":5.5499999999999998,"15min":5.5499999999999998},"max":{"1min":18,"5min":189.40199999999999,"15min":189.40199999999999},"last":15.994999999999999}]},{"osd":3,"last update":"Sat Jul 29 14:05:46 2023","interfaces":[{"interface":"back","average":{"1min":11.444000000000001,"5min":15.523999999999999,"15min":521.29999999999995},"min":{"1min":7.2489999999999997,"5min":6.335,"15min":6.335},"max":{"1min":37.712000000000003,"5min":187.852,"15min":30710.619999999999},"last":12.893000000000001},{"interface":"front","average":{"1min":15.505000000000001,"5min":18.006,"15min":520.84199999999998},"min":{"1min":10.118,"5min":8.8770000000000007,"15min":7.6180000000000003},"max":{"1min":38.539999999999999,"5min":101.88200000000001,"15min":30591.776999999998},"last":16.751000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:05:56 2023","interfaces":[{"interface":"back","average":{"1min":17.323,"5min":807.66700000000003,"15min":1018.088},"min":{"1min":8.2870000000000008,"5min":6.718,"15min":6.1070000000000002},"max":{"1min":70.116,"5min":25620.438999999998,"15min":32899.582000000002},"last":11.134},{"interface":"front","average":{"1min":17.379999999999999,"5min":826.65300000000002,"15min":1008.728},"min":{"1min":9.407,"5min":6.2089999999999996,"15min":6.2089999999999996},"max":{"1min":74.122,"5min":25979.712,"15min":32485.278999999999},"last":18.248000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:06:08 2023","interfaces":[{"interface":"back","average":{"1min":21.344999999999999,"5min":21.481000000000002,"15min":24.541},"min":{"1min":10.964,"5min":10.029,"15min":8.9290000000000003},"max":{"1min":94.355000000000004,"5min":94.355000000000004,"15min":189.423},"last":10.265000000000001},{"interface":"front","average":{"1min":22.614999999999998,"5min":18.805,"15min":21.620000000000001},"min":{"1min":10.656000000000001,"5min":7.3330000000000002,"15min":7.3330000000000002},"max":{"1min":91.700999999999993,"5min":91.700999999999993,"15min":188.518},"last":17.460999999999999}]},{"osd":7,"last update":"Sat Jul 29 14:05:46 2023","interfaces":[{"interface":"back","average":{"1min":15.869,"5min":19.315000000000001,"15min":19.748999999999999},"min":{"1min":12.207000000000001,"5min":7.1059999999999999,"15min":7.1059999999999999},"max":{"1min":19.248999999999999,"5min":74.337000000000003,"15min":200.55099999999999},"last":15.298999999999999},{"interface":"front","average":{"1min":14.26,"5min":19.501999999999999,"15min":21.978000000000002},"min":{"1min":9.8089999999999993,"5min":8.0790000000000006,"15min":8.0790000000000006},"max":{"1min":19.190999999999999,"5min":89.125,"15min":222.65199999999999},"last":19.901}]}]},{"osd":5,"up_from":704,"seq":3023656976779,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77876,"kb_used_data":6636,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94293964,"statfs":{"total":96636764160,"available":96557019136,"internally_reserved":0,"allocated":6795264,"data_stored":3844748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:04:00 2023","interfaces":[{"interface":"back","average":{"1min":23.48,"5min":22.623000000000001,"15min":41.000999999999998},"min":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":4.742},"max":{"1min":150.63,"5min":262.56099999999998,"15min":299.87799999999999},"last":7.3380000000000001},{"interface":"front","average":{"1min":24.837,"5min":22.727,"15min":52.545999999999999},"min":{"1min":5.8259999999999996,"5min":4.6539999999999999,"15min":4.6539999999999999},"max":{"1min":111.14100000000001,"5min":253.80000000000001,"15min":370.00999999999999},"last":14.272}]},{"osd":1,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":10.637,"5min":22.524000000000001,"15min":31.143000000000001},"min":{"1min":7.0529999999999999,"5min":5.3780000000000001,"15min":5.3780000000000001},"max":{"1min":18.199999999999999,"5min":362.58800000000002,"15min":362.58800000000002},"last":10.961},{"interface":"front","average":{"1min":12.446,"5min":30.510999999999999,"15min":52.337000000000003},"min":{"1min":7.0510000000000002,"5min":6.8680000000000003,"15min":6.8680000000000003},"max":{"1min":23.666,"5min":356.69900000000001,"15min":363.50999999999999},"last":15.521000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":13.893000000000001,"5min":21.827000000000002,"15min":47.597000000000001},"min":{"1min":6.0259999999999998,"5min":6.0259999999999998,"15min":6.0259999999999998},"max":{"1min":25.013999999999999,"5min":312.755,"15min":384.024},"last":8.3450000000000006},{"interface":"front","average":{"1min":14.029999999999999,"5min":16.576000000000001,"15min":24.143999999999998},"min":{"1min":8.4190000000000005,"5min":7.2759999999999998,"15min":7.2709999999999999},"max":{"1min":22.393999999999998,"5min":210.506,"15min":228.55600000000001},"last":11.526}]},{"osd":3,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":12.836,"5min":25.149000000000001,"15min":51.404000000000003},"min":{"1min":7.4240000000000004,"5min":7.4100000000000001,"15min":7.4100000000000001},"max":{"1min":19.074999999999999,"5min":312.12200000000001,"15min":381.23200000000003},"last":9.2110000000000003},{"interface":"front","average":{"1min":14.856999999999999,"5min":23.690000000000001,"15min":61.159999999999997},"min":{"1min":8.9109999999999996,"5min":7.944,"15min":7.944},"max":{"1min":24.484999999999999,"5min":263.30200000000002,"15min":382.779},"last":10.199999999999999}]},{"osd":4,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":17.190000000000001,"5min":1106.981,"15min":408.33800000000002},"min":{"1min":8.423,"5min":7.343,"15min":7.3070000000000004},"max":{"1min":42.877000000000002,"5min":23572.741999999998,"15min":23572.741999999998},"last":17.712},{"interface":"front","average":{"1min":14.066000000000001,"5min":1111.7170000000001,"15min":422.97000000000003},"min":{"1min":8.3000000000000007,"5min":6.8929999999999998,"15min":6.8929999999999998},"max":{"1min":25.968,"5min":23610.142,"15min":23610.142},"last":16.151}]},{"osd":6,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":14.585000000000001,"5min":21.475000000000001,"15min":47.896000000000001},"min":{"1min":7.5540000000000003,"5min":7.5540000000000003,"15min":7.5540000000000003},"max":{"1min":25.594000000000001,"5min":263.983,"15min":450.17500000000001},"last":12.238},{"interface":"front","average":{"1min":17.135999999999999,"5min":23.904,"15min":53.079000000000001},"min":{"1min":12.586,"5min":8.1969999999999992,"15min":8.1969999999999992},"max":{"1min":29.440999999999999,"5min":202.196,"15min":417.303},"last":13.643000000000001}]},{"osd":7,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":17.875,"5min":24.571000000000002,"15min":44.404000000000003},"min":{"1min":12.962999999999999,"5min":11.222,"15min":10.602},"max":{"1min":28.838000000000001,"5min":284.48000000000002,"15min":341.30500000000001},"last":18.501999999999999},{"interface":"front","average":{"1min":16.516999999999999,"5min":24.103999999999999,"15min":50.014000000000003},"min":{"1min":9.5359999999999996,"5min":9.5359999999999996,"15min":9.5359999999999996},"max":{"1min":28.317,"5min":176.96299999999999,"15min":329.86500000000001},"last":14.917}]}]},{"osd":7,"up_from":650,"seq":2791728742807,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59436,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312404,"statfs":{"total":96636764160,"available":96575901696,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:45 2023","interfaces":[{"interface":"back","average":{"1min":11.94,"5min":13.635,"15min":23.015000000000001},"min":{"1min":5.5890000000000004,"5min":5.4539999999999997,"15min":5.4539999999999997},"max":{"1min":28.5,"5min":75.968000000000004,"15min":293.94499999999999},"last":16.959},{"interface":"front","average":{"1min":14.968999999999999,"5min":18.658999999999999,"15min":28.620000000000001},"min":{"1min":6.8179999999999996,"5min":6.8179999999999996,"15min":6.8179999999999996},"max":{"1min":29.143999999999998,"5min":116.952,"15min":358.267},"last":22.059000000000001}]},{"osd":1,"last update":"Sat Jul 29 14:05:54 2023","interfaces":[{"interface":"back","average":{"1min":9.4100000000000001,"5min":1105.6959999999999,"15min":387.80799999999999},"min":{"1min":6.4740000000000002,"5min":5.1219999999999999,"15min":4.6660000000000004},"max":{"1min":13.821,"5min":27546.963,"15min":27546.963},"last":12.196999999999999},{"interface":"front","average":{"1min":12.48,"5min":1109.259,"15min":384.47300000000001},"min":{"1min":6.4139999999999997,"5min":6.4139999999999997,"15min":5.5010000000000003},"max":{"1min":16.233000000000001,"5min":27546.280999999999,"15min":27546.280999999999},"last":14.042999999999999}]},{"osd":2,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":13.449999999999999,"5min":17.512,"15min":25.91},"min":{"1min":9.0869999999999997,"5min":9.0869999999999997,"15min":7.9100000000000001},"max":{"1min":15.819000000000001,"5min":103.09,"15min":461.83800000000002},"last":11.837999999999999},{"interface":"front","average":{"1min":8.734,"5min":13.756,"15min":21.891999999999999},"min":{"1min":7.0039999999999996,"5min":6.6239999999999997,"15min":6.6239999999999997},"max":{"1min":13.706,"5min":181.97900000000001,"15min":385.40800000000002},"last":7.4480000000000004}]},{"osd":3,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":10.454000000000001,"5min":17.827999999999999,"15min":1298.6089999999999},"min":{"1min":7.8300000000000001,"5min":7.6660000000000004,"15min":7.6660000000000004},"max":{"1min":16.02,"5min":242.197,"15min":34550.838000000003},"last":10.022},{"interface":"front","average":{"1min":15.295,"5min":23.765999999999998,"15min":1309.0609999999999},"min":{"1min":12.576000000000001,"5min":8.1509999999999998,"15min":6.9550000000000001},"max":{"1min":18.369,"5min":243.06399999999999,"15min":34556.381000000001},"last":18.600000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:05:54 2023","interfaces":[{"interface":"back","average":{"1min":11.284000000000001,"5min":1071.366,"15min":900.10199999999998},"min":{"1min":6.1319999999999997,"5min":6.1319999999999997,"15min":6.1319999999999997},"max":{"1min":18.895,"5min":25146.41,"15min":27891.68},"last":14.632},{"interface":"front","average":{"1min":15.611000000000001,"5min":1099.066,"15min":916.32100000000003},"min":{"1min":7.5990000000000002,"5min":7.5990000000000002,"15min":7.2699999999999996},"max":{"1min":21.300999999999998,"5min":25434.151999999998,"15min":27883.419999999998},"last":20.539000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:06:08 2023","interfaces":[{"interface":"back","average":{"1min":17.609000000000002,"5min":23.239999999999998,"15min":66.055999999999997},"min":{"1min":13.454000000000001,"5min":11.215,"15min":10.683},"max":{"1min":23.506,"5min":149.554,"15min":1265.123},"last":17.620999999999999},{"interface":"front","average":{"1min":12.923,"5min":22.481000000000002,"15min":69.558999999999997},"min":{"1min":6.7140000000000004,"5min":6.7140000000000004,"15min":6.7140000000000004},"max":{"1min":20.945,"5min":255.23699999999999,"15min":1234.9300000000001},"last":10.840999999999999}]},{"osd":6,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":15.662000000000001,"5min":20.847000000000001,"15min":24.681000000000001},"min":{"1min":11.005000000000001,"5min":9.9740000000000002,"15min":7.8689999999999998},"max":{"1min":49.978000000000002,"5min":95.558999999999997,"15min":243.66499999999999},"last":11.486000000000001},{"interface":"front","average":{"1min":21.350999999999999,"5min":25.521999999999998,"15min":30.802},"min":{"1min":16.145,"5min":12.111000000000001,"15min":10.542},"max":{"1min":79.692999999999998,"5min":148.334,"15min":454.68799999999999},"last":21.289999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-29T14:06:16.380 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:06:16.381 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:06:16.381 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:06:16.382 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:06:16.382 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:06:16.383 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:06:16.383 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2023-07-29T13:29:15.987740+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=13, tm_min=29, tm_sec=15, tm_wday=5, tm_yday=210, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=29, tm_hour=14, tm_min=5, tm_sec=47, tm_wday=5, tm_yday=210, tm_isdst=0) 2023-07-29T14:06:16.383 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-07-29T14:06:27.153 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:22.488 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:27.160 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:22.494 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:27.181 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:22.515 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:27.213 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:22.547 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:27.214 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:22.495 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:27.214 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:22.498 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:27.216 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:22.552 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:27.229 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:22.515 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:27.230 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:22.566 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:32.170 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:27.505 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:32.175 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:27.510 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:32.187 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:27.523 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:32.222 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:27.557 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:32.223 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:27.508 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:32.226 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:27.511 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:32.229 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:27.563 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:32.236 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:27.522 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:32.239 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:27.575 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:36.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-29T14:06:36.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.504+0000 7f54f81f2640 10 monclient: get_monmap_and_config 2023-07-29T14:06:36.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.504+0000 7f54f81f2640 10 monclient: build_initial_monmap 2023-07-29T14:06:36.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.504+0000 7f54f81f2640 10 monclient: monmap: 2023-07-29T14:06:36.511 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:06:36.511 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:36.511 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:06:36.505931+0000 2023-07-29T14:06:36.511 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:06:36.505931+0000 2023-07-29T14:06:36.511 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:06:36.511 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:36.511 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:06:36.512 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:06:36.512 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:06:36.512 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:06:36.512 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:06:36.513 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:36.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.505+0000 7f54f81f2640 1 Processor -- start 2023-07-29T14:06:36.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.505+0000 7f54f81f2640 1 -- start start 2023-07-29T14:06:36.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.505+0000 7f54f81f2640 10 monclient: init 2023-07-29T14:06:36.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 10 monclient: _reopen_session rank -1 2023-07-29T14:06:36.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2023-07-29T14:06:36.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f54f00b74a0 addr v1:172.21.15.45:6789/0 2023-07-29T14:06:36.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54f013e4f0 0x7f54f00aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 10 monclient(hunting): picked mon.noname-c con 0x7f54f013e4f0 addr v2:172.21.15.45:3300/0 2023-07-29T14:06:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f01445e0 0x7f54f0144a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 10 monclient(hunting): picked mon.noname-b con 0x7f54f01445e0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:06:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f00b7870 con 0x7f54f00b74a0 2023-07-29T14:06:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f00a91b0 con 0x7f54f013e4f0 2023-07-29T14:06:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:36.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00a9490 con 0x7f54f01445e0 2023-07-29T14:06:36.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 10 monclient(hunting): _renew_subs 2023-07-29T14:06:36.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f81f2640 10 monclient(hunting): authenticate will time out at 3718.639230s 2023-07-29T14:06:36.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f678f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f01445e0 0x7f54f0144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:36.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.506+0000 7f54f578d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54f013e4f0 0x7f54f00aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:36.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f678f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f01445e0 0x7f54f0144a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35292/0 (socket says 172.21.15.40:35292) 2023-07-29T14:06:36.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f678f640 1 -- 172.21.15.40:0/1617124329 learned_addr learned my addr 172.21.15.40:0/1617124329 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:06:36.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f678f640 10 monclient(hunting): get_auth_request con 0x7f54f01445e0 auth_method 0 2023-07-29T14:06:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f678f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f678f640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f678f640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f578d640 10 monclient(hunting): get_auth_request con 0x7f54f013e4f0 auth_method 0 2023-07-29T14:06:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f578d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f578d640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f578d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f678f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f678f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.507+0000 7f54f678f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.508+0000 7f54f578d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.508+0000 7f54f578d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.508+0000 7f54f578d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.509+0000 7f54f678f640 10 monclient(hunting): handle_auth_done global_id 16448 payload 274 2023-07-29T14:06:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.509+0000 7f54f678f640 1 -- 172.21.15.40:0/1617124329 >> v2:172.21.15.45:3300/0 conn(0x7f54f013e4f0 msgr2=0x7f54f00aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:06:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.509+0000 7f54f678f640 1 --2- 172.21.15.40:0/1617124329 >> v2:172.21.15.45:3300/0 conn(0x7f54f013e4f0 0x7f54f00aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.509+0000 7f54f678f640 1 -- 172.21.15.40:0/1617124329 >> v1:172.21.15.45:6789/0 conn(0x7f54f00b74a0 legacy=0x7f54f013dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.509+0000 7f54f678f640 10 monclient: _finish_hunting 0 2023-07-29T14:06:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.509+0000 7f54f678f640 1 monclient: found mon.noname-b 2023-07-29T14:06:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.509+0000 7f54f678f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:06:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.509+0000 7f54f678f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:06:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.509+0000 7f54f678f640 1 -- 172.21.15.40:0/1617124329 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f00a9770 con 0x7f54f01445e0 2023-07-29T14:06:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.511+0000 7f54f678f640 1 --2- 172.21.15.40:0/1617124329 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f01445e0 0x7f54f0144a60 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009e50 tx=0x7f54e802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b67770c3dd3dad1 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:06:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 1 -- 172.21.15.40:0/1617124329 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e802ca70 con 0x7f54f01445e0 2023-07-29T14:06:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:06:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 10 monclient: dump: 2023-07-29T14:06:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:36.537 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 10 monclient: _finish_auth 0 2023-07-29T14:06:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 10 monclient: _check_auth_tickets 2023-07-29T14:06:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:06:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 1 -- 172.21.15.40:0/1617124329 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e802cbd0 con 0x7f54f01445e0 2023-07-29T14:06:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f81f2640 5 monclient: authenticate success, global_id 16448 2023-07-29T14:06:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f81f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-07-29T14:06:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:06:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f81f2640 10 monclient: get_monmap_and_config success 2023-07-29T14:06:36.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f81f2640 4 set_mon_vals no callback set 2023-07-29T14:06:36.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 1 -- 172.21.15.40:0/1617124329 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e80343f0 con 0x7f54f01445e0 2023-07-29T14:06:36.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:36.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:06:36.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f4f8c640 10 monclient: dump: 2023-07-29T14:06:36.543 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:36.543 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:36.543 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:36.543 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:36.543 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:36.543 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:36.543 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:36.544 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f81f2640 10 monclient: shutdown 2023-07-29T14:06:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f81f2640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:06:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f81f2640 1 -- 172.21.15.40:0/1617124329 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f01445e0 msgr2=0x7f54f0144a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.516+0000 7f54f81f2640 1 --2- 172.21.15.40:0/1617124329 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f01445e0 0x7f54f0144a60 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009e50 tx=0x7f54e802bed0 comp rx=0 tx=0).stop 2023-07-29T14:06:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.517+0000 7f54f81f2640 1 -- 172.21.15.40:0/1617124329 shutdown_connections 2023-07-29T14:06:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.517+0000 7f54f81f2640 1 --2- 172.21.15.40:0/1617124329 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f01445e0 0x7f54f0144a60 unknown :-1 s=CLOSED pgs=3244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.517+0000 7f54f81f2640 1 --2- 172.21.15.40:0/1617124329 >> v2:172.21.15.45:3300/0 conn(0x7f54f013e4f0 0x7f54f00aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.517+0000 7f54f81f2640 1 -- 172.21.15.40:0/1617124329 >> 172.21.15.40:0/1617124329 conn(0x7f54f0065b00 msgr2=0x7f54f0065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:06:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.517+0000 7f54f81f2640 1 -- 172.21.15.40:0/1617124329 shutdown_connections 2023-07-29T14:06:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.517+0000 7f54f81f2640 1 -- 172.21.15.40:0/1617124329 wait complete. 2023-07-29T14:06:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.517+0000 7f54f81f2640 10 monclient: build_initial_monmap 2023-07-29T14:06:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.517+0000 7f54f81f2640 10 monclient: monmap: 2023-07-29T14:06:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:06:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:06:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:06:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:06:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:06:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:06:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:06:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:06:36.548 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:36.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.518+0000 7f54f81f2640 1 librados: starting msgr at 2023-07-29T14:06:36.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.518+0000 7f54f81f2640 1 librados: starting objecter 2023-07-29T14:06:36.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.518+0000 7f54f81f2640 1 Processor -- start 2023-07-29T14:06:36.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.518+0000 7f54f81f2640 1 -- start start 2023-07-29T14:06:36.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.518+0000 7f54f81f2640 1 librados: setting wanted keys 2023-07-29T14:06:36.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.518+0000 7f54f81f2640 1 librados: calling monclient init 2023-07-29T14:06:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.518+0000 7f54f81f2640 10 monclient: init 2023-07-29T14:06:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.518+0000 7f54f81f2640 10 monclient: _reopen_session rank -1 2023-07-29T14:06:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.518+0000 7f54f81f2640 10 monclient: _add_conns ranks=[2,0,1] 2023-07-29T14:06:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.518+0000 7f54f81f2640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f00b74a0 0x7f54f00b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 10 monclient(hunting): picked mon.noname-c con 0x7f54f00b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:06:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f013e4f0 0x7f54f00ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 10 monclient(hunting): picked mon.noname-a con 0x7f54f013e4f0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:06:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f01445e0 0x7f54f00ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 10 monclient(hunting): picked mon.noname-b con 0x7f54f01445e0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:06:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00aed20 con 0x7f54f013e4f0 2023-07-29T14:06:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f0143d80 con 0x7f54f01445e0 2023-07-29T14:06:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00b0580 con 0x7f54f00b74a0 2023-07-29T14:06:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 10 monclient(hunting): _renew_subs 2023-07-29T14:06:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f81f2640 10 monclient(hunting): authenticate will time out at 3718.652127s 2023-07-29T14:06:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f5f8e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f00b74a0 0x7f54f00b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f578d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f013e4f0 0x7f54f00ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f578d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f013e4f0 0x7f54f00ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49626/0 (socket says 172.21.15.40:49626) 2023-07-29T14:06:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f578d640 1 -- 172.21.15.40:0/1835232061 learned_addr learned my addr 172.21.15.40:0/1835232061 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:06:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f578d640 10 monclient(hunting): get_auth_request con 0x7f54f013e4f0 auth_method 0 2023-07-29T14:06:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f578d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f578d640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f578d640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f678f640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f01445e0 0x7f54f00ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f5f8e640 10 monclient(hunting): get_auth_request con 0x7f54f00b74a0 auth_method 0 2023-07-29T14:06:36.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f5f8e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:36.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f5f8e640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:36.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.519+0000 7f54f5f8e640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:36.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f678f640 10 monclient(hunting): get_auth_request con 0x7f54f01445e0 auth_method 0 2023-07-29T14:06:36.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f678f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:36.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f678f640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:36.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f678f640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:36.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f5f8e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:36.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f5f8e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:36.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f5f8e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:36.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f578d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f578d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:36.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f578d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:36.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f678f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:36.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f678f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:36.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.520+0000 7f54f678f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:36.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.523+0000 7f54f5f8e640 10 monclient(hunting): handle_auth_done global_id 16451 payload 824 2023-07-29T14:06:36.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.523+0000 7f54f5f8e640 1 -- 172.21.15.40:0/1835232061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f01445e0 msgr2=0x7f54f00ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:36.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.523+0000 7f54f5f8e640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f01445e0 0x7f54f00ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:36.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.523+0000 7f54f5f8e640 1 -- 172.21.15.40:0/1835232061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f013e4f0 msgr2=0x7f54f00ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:36.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.523+0000 7f54f5f8e640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f013e4f0 0x7f54f00ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:36.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.523+0000 7f54f5f8e640 10 monclient: _finish_hunting 0 2023-07-29T14:06:36.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.523+0000 7f54f5f8e640 1 monclient: found mon.noname-c 2023-07-29T14:06:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.523+0000 7f54f5f8e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:06:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.523+0000 7f54f5f8e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:06:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.523+0000 7f54f5f8e640 1 -- 172.21.15.40:0/1835232061 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e8009b30 con 0x7f54f00b74a0 2023-07-29T14:06:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.527+0000 7f54f5f8e640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f00b74a0 0x7f54f00b0ca0 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c3c0 tx=0x7f54e000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:06:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.527+0000 7f54e67fc640 10 client.?.objecter ms_handle_connect 0x7f54f00b74a0 2023-07-29T14:06:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.527+0000 7f54e67fc640 10 client.?.objecter resend_mon_ops 2023-07-29T14:06:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 1 -- 172.21.15.40:0/1835232061 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e00052d0 con 0x7f54f00b74a0 2023-07-29T14:06:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:06:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 10 monclient: dump: 2023-07-29T14:06:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:36.566 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:36.566 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:36.566 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:36.566 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:36.566 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:36.566 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:36.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 10 monclient: _finish_auth 0 2023-07-29T14:06:36.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 10 monclient: _check_auth_tickets 2023-07-29T14:06:36.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:06:36.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 1 -- 172.21.15.40:0/1835232061 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e0005430 con 0x7f54f00b74a0 2023-07-29T14:06:36.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:06:36.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 1 -- 172.21.15.40:0/1835232061 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0006a60 con 0x7f54f00b74a0 2023-07-29T14:06:36.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:36.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:06:36.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e67fc640 10 monclient: dump: 2023-07-29T14:06:36.568 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:36.568 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:36.568 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:36.568 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:36.568 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:36.568 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:36.569 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:36.569 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:36.569 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:36.569 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:36.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54f81f2640 5 monclient: authenticate success, global_id 16451 2023-07-29T14:06:36.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54f81f2640 10 monclient: _renew_subs 2023-07-29T14:06:36.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54e7fff640 4 set_mon_vals no callback set 2023-07-29T14:06:36.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54f81f2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:06:36.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54f81f2640 1 -- 172.21.15.40:0/1835232061 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f00b3910 con 0x7f54f00b74a0 2023-07-29T14:06:36.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54f81f2640 10 client.16451.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:06:36.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54f81f2640 10 monclient: _renew_subs 2023-07-29T14:06:36.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54f81f2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:06:36.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54f81f2640 1 -- 172.21.15.40:0/1835232061 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f00b4730 con 0x7f54f00b74a0 2023-07-29T14:06:36.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.531+0000 7f54f81f2640 1 librados: init done 2023-07-29T14:06:36.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.532+0000 7f54f81f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:06:36.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.532+0000 7f54f81f2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:06:36.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.532+0000 7f54f81f2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:06:36.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.532+0000 7f54f81f2640 1 -- 172.21.15.40:0/1835232061 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f004e000 con 0x7f54f00b74a0 2023-07-29T14:06:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.793+0000 7f54e67fc640 1 -- 172.21.15.40:0/1835232061 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f54e0003640 con 0x7f54f00b74a0 2023-07-29T14:06:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.793+0000 7f54e67fc640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f54d406bfb0 0x7f54d406e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.794+0000 7f54f578d640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f54d406bfb0 0x7f54d406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.794+0000 7f54f578d640 10 monclient: get_auth_request con 0x7f54d406bfb0 auth_method 0 2023-07-29T14:06:36.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.794+0000 7f54f578d640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f54d406bfb0 0x7f54d406e460 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f54f00adee0 tx=0x7f54ec009190 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:06:36.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.794+0000 7f54e67fc640 10 client.16451.objecter ms_handle_connect 0x7f54d406bfb0 2023-07-29T14:06:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.806+0000 7f54e67fc640 1 -- 172.21.15.40:0/1835232061 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f54e005b770 con 0x7f54f00b74a0 2023-07-29T14:06:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.806+0000 7f54e67fc640 10 client.16451.objecter ms_dispatch 0x7f54f0000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:06:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.806+0000 7f54e67fc640 3 client.16451.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:06:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.806+0000 7f54e67fc640 3 client.16451.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:06:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.806+0000 7f54e67fc640 20 client.16451.objecter dump_active .. 0 homeless 2023-07-29T14:06:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.806+0000 7f54e67fc640 1 -- 172.21.15.40:0/1835232061 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f54e0053ba0 con 0x7f54f00b74a0 2023-07-29T14:06:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.806+0000 7f54e67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:06:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.806+0000 7f54e67fc640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:06:36.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.982+0000 7f54f81f2640 1 -- 172.21.15.40:0/1835232061 --> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54f00aa750 con 0x7f54d406bfb0 2023-07-29T14:06:36.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.987+0000 7f54e67fc640 1 -- 172.21.15.40:0/1835232061 <== mgr.4152 v2:172.21.15.45:6800/84611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152658 (secure 0 0 0) 0x7f54ec011af0 con 0x7f54d406bfb0 2023-07-29T14:06:36.989 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-29T14:06:36.990 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-29T14:06:36.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.989+0000 7f54f81f2640 10 librados: watch_flush enter 2023-07-29T14:06:36.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 10 librados: watch_flush exit 2023-07-29T14:06:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 20 client.16451.objecter shutdown clearing up homeless session... 2023-07-29T14:06:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 10 client.16451.objecter successfully canceled tick 2023-07-29T14:06:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 -- 172.21.15.40:0/1835232061 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f54d406bfb0 msgr2=0x7f54d406e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:37.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f54d406bfb0 0x7f54d406e460 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f54f00adee0 tx=0x7f54ec009190 comp rx=0 tx=0).stop 2023-07-29T14:06:37.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 10 monclient: shutdown 2023-07-29T14:06:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:06:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 -- 172.21.15.40:0/1835232061 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f00b74a0 msgr2=0x7f54f00b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f00b74a0 0x7f54f00b0ca0 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c3c0 tx=0x7f54e000c880 comp rx=0 tx=0).stop 2023-07-29T14:06:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 -- 172.21.15.40:0/1835232061 shutdown_connections 2023-07-29T14:06:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f54d406bfb0 0x7f54d406e460 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f01445e0 0x7f54f00ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f013e4f0 0x7f54f00ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 --2- 172.21.15.40:0/1835232061 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f54f00b74a0 0x7f54f00b0ca0 unknown :-1 s=CLOSED pgs=3245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 -- 172.21.15.40:0/1835232061 >> 172.21.15.40:0/1835232061 conn(0x7f54f0065b00 msgr2=0x7f54f00b61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:06:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 -- 172.21.15.40:0/1835232061 shutdown_connections 2023-07-29T14:06:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 -- 172.21.15.40:0/1835232061 wait complete. 2023-07-29T14:06:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:36.990+0000 7f54f81f2640 1 librados: shutdown 2023-07-29T14:06:37.006 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":2159,"stamp":"2023-07-29T14:06:35.641684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":520568,"kb_used_data":50744,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754454152,"statfs":{"total":773094113280,"available":772561051648,"internally_reserved":0,"allocated":51961856,"data_stored":28370954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002771"},"pg_stats":[{"pgid":"7.1b","version":"0'0","reported_seq":981,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:00.811211+0000","last_change":"2023-07-29T14:06:00.817006+0000","last_active":"2023-07-29T14:06:00.811211+0000","last_peered":"2023-07-29T14:06:00.811211+0000","last_clean":"2023-07-29T14:06:00.811211+0000","last_became_active":"2023-07-29T13:56:55.656930+0000","last_became_peered":"2023-07-29T13:56:55.656930+0000","last_unstale":"2023-07-29T14:06:00.811211+0000","last_undegraded":"2023-07-29T14:06:00.811211+0000","last_fullsized":"2023-07-29T14:06:00.811211+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:00.810385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:00.810385+0000","last_clean_scrub_stamp":"2023-07-29T14:06:00.810385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:58:40.689965+0000","scrub_duration":0.070146252000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":990,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:07.186288+0000","last_change":"2023-07-29T14:06:07.190582+0000","last_active":"2023-07-29T14:06:07.186288+0000","last_peered":"2023-07-29T14:06:07.186288+0000","last_clean":"2023-07-29T14:06:07.186288+0000","last_became_active":"2023-07-29T13:41:38.322679+0000","last_became_peered":"2023-07-29T13:41:38.322679+0000","last_unstale":"2023-07-29T14:06:07.186288+0000","last_undegraded":"2023-07-29T14:06:07.186288+0000","last_fullsized":"2023-07-29T14:06:07.186288+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:07.185413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:07.185413+0000","last_clean_scrub_stamp":"2023-07-29T14:06:07.185413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:26:55.323205+0000","scrub_duration":0.040433750999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":978,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:06.240995+0000","last_change":"2023-07-29T14:06:06.245535+0000","last_active":"2023-07-29T14:06:06.240995+0000","last_peered":"2023-07-29T14:06:06.240995+0000","last_clean":"2023-07-29T14:06:06.240995+0000","last_became_active":"2023-07-29T13:29:52.099163+0000","last_became_peered":"2023-07-29T13:29:52.099163+0000","last_unstale":"2023-07-29T14:06:06.240995+0000","last_undegraded":"2023-07-29T14:06:06.240995+0000","last_fullsized":"2023-07-29T14:06:06.240995+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:06.240199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:06.240199+0000","last_clean_scrub_stamp":"2023-07-29T14:06:06.240199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:43:50.331024+0000","scrub_duration":0.050444346000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":944,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:54.861717+0000","last_change":"2023-07-29T14:05:54.863494+0000","last_active":"2023-07-29T14:05:54.861717+0000","last_peered":"2023-07-29T14:05:54.861717+0000","last_clean":"2023-07-29T14:05:54.861717+0000","last_became_active":"2023-07-29T13:29:41.782506+0000","last_became_peered":"2023-07-29T13:29:41.782506+0000","last_unstale":"2023-07-29T14:05:54.861717+0000","last_undegraded":"2023-07-29T14:05:54.861717+0000","last_fullsized":"2023-07-29T14:05:54.861717+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:54.860920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:54.860920+0000","last_clean_scrub_stamp":"2023-07-29T14:05:54.860920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:14:42.977652+0000","scrub_duration":0.035476684000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":960,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:49.198542+0000","last_change":"2023-07-29T14:05:49.205817+0000","last_active":"2023-07-29T14:05:49.198542+0000","last_peered":"2023-07-29T14:05:49.198542+0000","last_clean":"2023-07-29T14:05:49.198542+0000","last_became_active":"2023-07-29T14:05:38.060277+0000","last_became_peered":"2023-07-29T14:05:38.060277+0000","last_unstale":"2023-07-29T14:05:49.198542+0000","last_undegraded":"2023-07-29T14:05:49.198542+0000","last_fullsized":"2023-07-29T14:05:49.198542+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:49.197193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:49.197193+0000","last_clean_scrub_stamp":"2023-07-29T14:05:49.197193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:55:48.982266+0000","scrub_duration":0.090687935999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":927,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:14.022387+0000","last_change":"2023-07-29T14:06:14.024200+0000","last_active":"2023-07-29T14:06:14.022387+0000","last_peered":"2023-07-29T14:06:14.022387+0000","last_clean":"2023-07-29T14:06:14.022387+0000","last_became_active":"2023-07-29T13:53:19.600386+0000","last_became_peered":"2023-07-29T13:53:19.600386+0000","last_unstale":"2023-07-29T14:06:14.022387+0000","last_undegraded":"2023-07-29T14:06:14.022387+0000","last_fullsized":"2023-07-29T14:06:14.022387+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:14.021538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:14.021538+0000","last_clean_scrub_stamp":"2023-07-29T14:06:14.021538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:51:32.345412+0000","scrub_duration":0.060272523000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":949,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:04.307469+0000","last_change":"2023-07-29T14:06:04.309240+0000","last_active":"2023-07-29T14:06:04.307469+0000","last_peered":"2023-07-29T14:06:04.307469+0000","last_clean":"2023-07-29T14:06:04.307469+0000","last_became_active":"2023-07-29T13:56:55.400591+0000","last_became_peered":"2023-07-29T13:56:55.400591+0000","last_unstale":"2023-07-29T14:06:04.307469+0000","last_undegraded":"2023-07-29T14:06:04.307469+0000","last_fullsized":"2023-07-29T14:06:04.307469+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:04.306657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:04.306657+0000","last_clean_scrub_stamp":"2023-07-29T14:06:04.306657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:17:55.974071+0000","scrub_duration":0.081655448000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":967,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:53.019638+0000","last_change":"2023-07-29T14:05:53.021396+0000","last_active":"2023-07-29T14:05:53.019638+0000","last_peered":"2023-07-29T14:05:53.019638+0000","last_clean":"2023-07-29T14:05:53.019638+0000","last_became_active":"2023-07-29T13:29:31.072042+0000","last_became_peered":"2023-07-29T13:29:31.072042+0000","last_unstale":"2023-07-29T14:05:53.019638+0000","last_undegraded":"2023-07-29T14:05:53.019638+0000","last_fullsized":"2023-07-29T14:05:53.019638+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:53.018835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:53.018835+0000","last_clean_scrub_stamp":"2023-07-29T14:05:53.018835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:48:12.562623+0000","scrub_duration":0.032764568000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":926,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:15.057094+0000","last_change":"2023-07-29T14:06:15.062870+0000","last_active":"2023-07-29T14:06:15.057094+0000","last_peered":"2023-07-29T14:06:15.057094+0000","last_clean":"2023-07-29T14:06:15.057094+0000","last_became_active":"2023-07-29T13:53:19.671215+0000","last_became_peered":"2023-07-29T13:53:19.671215+0000","last_unstale":"2023-07-29T14:06:15.057094+0000","last_undegraded":"2023-07-29T14:06:15.057094+0000","last_fullsized":"2023-07-29T14:06:15.057094+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:15.056251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:15.056251+0000","last_clean_scrub_stamp":"2023-07-29T14:06:15.056251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T15:58:45.678440+0000","scrub_duration":0.079510814999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":999,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:55.184407+0000","last_change":"2023-07-29T14:05:55.189991+0000","last_active":"2023-07-29T14:05:55.184407+0000","last_peered":"2023-07-29T14:05:55.184407+0000","last_clean":"2023-07-29T14:05:55.184407+0000","last_became_active":"2023-07-29T13:56:55.511490+0000","last_became_peered":"2023-07-29T13:56:55.511490+0000","last_unstale":"2023-07-29T14:05:55.184407+0000","last_undegraded":"2023-07-29T14:05:55.184407+0000","last_fullsized":"2023-07-29T14:05:55.184407+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:55.183058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:55.183058+0000","last_clean_scrub_stamp":"2023-07-29T14:05:55.183058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:12:19.154130+0000","scrub_duration":0.037892422000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":981,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:54.998356+0000","last_change":"2023-07-29T14:05:55.000139+0000","last_active":"2023-07-29T14:05:54.998356+0000","last_peered":"2023-07-29T14:05:54.998356+0000","last_clean":"2023-07-29T14:05:54.998356+0000","last_became_active":"2023-07-29T14:05:37.617822+0000","last_became_peered":"2023-07-29T14:05:37.617822+0000","last_unstale":"2023-07-29T14:05:54.998356+0000","last_undegraded":"2023-07-29T14:05:54.998356+0000","last_fullsized":"2023-07-29T14:05:54.998356+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:54.997537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:54.997537+0000","last_clean_scrub_stamp":"2023-07-29T14:05:54.997537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:05:05.354390+0000","scrub_duration":0.060110108000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":926,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:12.963049+0000","last_change":"2023-07-29T14:06:12.964828+0000","last_active":"2023-07-29T14:06:12.963049+0000","last_peered":"2023-07-29T14:06:12.963049+0000","last_clean":"2023-07-29T14:06:12.963049+0000","last_became_active":"2023-07-29T13:53:19.637473+0000","last_became_peered":"2023-07-29T13:53:19.637473+0000","last_unstale":"2023-07-29T14:06:12.963049+0000","last_undegraded":"2023-07-29T14:06:12.963049+0000","last_fullsized":"2023-07-29T14:06:12.963049+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:12.962193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:12.962193+0000","last_clean_scrub_stamp":"2023-07-29T14:06:12.962193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:04:10.539578+0000","scrub_duration":0.040380061000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":945,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:14.802832+0000","last_change":"2023-07-29T14:06:14.807191+0000","last_active":"2023-07-29T14:06:14.802832+0000","last_peered":"2023-07-29T14:06:14.802832+0000","last_clean":"2023-07-29T14:06:14.802832+0000","last_became_active":"2023-07-29T13:29:33.515255+0000","last_became_peered":"2023-07-29T13:29:33.515255+0000","last_unstale":"2023-07-29T14:06:14.802832+0000","last_undegraded":"2023-07-29T14:06:14.802832+0000","last_fullsized":"2023-07-29T14:06:14.802832+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:14.802005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:14.802005+0000","last_clean_scrub_stamp":"2023-07-29T14:06:14.802005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:17:41.718958+0000","scrub_duration":0.037248046,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":998,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:00.296691+0000","last_change":"2023-07-29T14:06:00.298559+0000","last_active":"2023-07-29T14:06:00.296691+0000","last_peered":"2023-07-29T14:06:00.296691+0000","last_clean":"2023-07-29T14:06:00.296691+0000","last_became_active":"2023-07-29T14:05:37.510437+0000","last_became_peered":"2023-07-29T14:05:37.510437+0000","last_unstale":"2023-07-29T14:06:00.296691+0000","last_undegraded":"2023-07-29T14:06:00.296691+0000","last_fullsized":"2023-07-29T14:06:00.296691+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:00.295845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:00.295845+0000","last_clean_scrub_stamp":"2023-07-29T14:06:00.295845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T22:40:35.629879+0000","scrub_duration":0.036103258999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":951,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:16.118492+0000","last_change":"2023-07-29T14:06:16.124056+0000","last_active":"2023-07-29T14:06:16.118492+0000","last_peered":"2023-07-29T14:06:16.118492+0000","last_clean":"2023-07-29T14:06:16.118492+0000","last_became_active":"2023-07-29T13:56:55.139167+0000","last_became_peered":"2023-07-29T13:56:55.139167+0000","last_unstale":"2023-07-29T14:06:16.118492+0000","last_undegraded":"2023-07-29T14:06:16.118492+0000","last_fullsized":"2023-07-29T14:06:16.118492+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:16.117687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:16.117687+0000","last_clean_scrub_stamp":"2023-07-29T14:06:16.117687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:02:30.187272+0000","scrub_duration":0.065360020000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":964,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:52.809995+0000","last_change":"2023-07-29T14:05:52.811915+0000","last_active":"2023-07-29T14:05:52.809995+0000","last_peered":"2023-07-29T14:05:52.809995+0000","last_clean":"2023-07-29T14:05:52.809995+0000","last_became_active":"2023-07-29T14:05:37.614802+0000","last_became_peered":"2023-07-29T14:05:37.614802+0000","last_unstale":"2023-07-29T14:05:52.809995+0000","last_undegraded":"2023-07-29T14:05:52.809995+0000","last_fullsized":"2023-07-29T14:05:52.809995+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:52.809166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:52.809166+0000","last_clean_scrub_stamp":"2023-07-29T14:05:52.809166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:08:42.706414+0000","scrub_duration":0.036979765999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1002,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:54.148401+0000","last_change":"2023-07-29T14:05:54.154623+0000","last_active":"2023-07-29T14:05:54.148401+0000","last_peered":"2023-07-29T14:05:54.148401+0000","last_clean":"2023-07-29T14:05:54.148401+0000","last_became_active":"2023-07-29T13:56:55.065724+0000","last_became_peered":"2023-07-29T13:56:55.065724+0000","last_unstale":"2023-07-29T14:05:54.148401+0000","last_undegraded":"2023-07-29T14:05:54.148401+0000","last_fullsized":"2023-07-29T14:05:54.148401+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:54.147587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:54.147587+0000","last_clean_scrub_stamp":"2023-07-29T14:05:54.147587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T19:33:58.287861+0000","scrub_duration":0.044510354000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":985,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:56.293729+0000","last_change":"2023-07-29T14:05:56.293729+0000","last_active":"2023-07-29T14:05:56.293729+0000","last_peered":"2023-07-29T14:05:56.293729+0000","last_clean":"2023-07-29T14:05:56.293729+0000","last_became_active":"2023-07-29T13:28:14.423044+0000","last_became_peered":"2023-07-29T13:28:14.423044+0000","last_unstale":"2023-07-29T14:05:56.293729+0000","last_undegraded":"2023-07-29T14:05:56.293729+0000","last_fullsized":"2023-07-29T14:05:56.293729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:56.291155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:56.291155+0000","last_clean_scrub_stamp":"2023-07-29T14:05:56.291155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:42:54.557575+0000","scrub_duration":0.032224926000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1","version":"1004'8","reported_seq":997,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:52.065768+0000","last_change":"2023-07-29T14:05:52.069559+0000","last_active":"2023-07-29T14:05:52.065768+0000","last_peered":"2023-07-29T14:05:52.065768+0000","last_clean":"2023-07-29T14:05:52.065768+0000","last_became_active":"2023-07-29T13:41:38.521847+0000","last_became_peered":"2023-07-29T13:41:38.521847+0000","last_unstale":"2023-07-29T14:05:52.065768+0000","last_undegraded":"2023-07-29T14:05:52.065768+0000","last_fullsized":"2023-07-29T14:05:52.065768+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"1004'8","last_scrub_stamp":"2023-07-29T14:05:52.064254+0000","last_deep_scrub":"1004'8","last_deep_scrub_stamp":"2023-07-29T14:05:52.064254+0000","last_clean_scrub_stamp":"2023-07-29T14:05:52.064254+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T01:31:35.837707+0000","scrub_duration":0.082755203999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":957,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:57.236762+0000","last_change":"2023-07-29T14:05:57.242910+0000","last_active":"2023-07-29T14:05:57.236762+0000","last_peered":"2023-07-29T14:05:57.236762+0000","last_clean":"2023-07-29T14:05:57.236762+0000","last_became_active":"2023-07-29T13:56:55.258241+0000","last_became_peered":"2023-07-29T13:56:55.258241+0000","last_unstale":"2023-07-29T14:05:57.236762+0000","last_undegraded":"2023-07-29T14:05:57.236762+0000","last_fullsized":"2023-07-29T14:05:57.236762+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:57.235949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:57.235949+0000","last_clean_scrub_stamp":"2023-07-29T14:05:57.235949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:52:01.628478+0000","scrub_duration":0.034914097999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":980,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:18.364524+0000","last_change":"2023-07-29T14:06:18.366357+0000","last_active":"2023-07-29T14:06:18.364524+0000","last_peered":"2023-07-29T14:06:18.364524+0000","last_clean":"2023-07-29T14:06:18.364524+0000","last_became_active":"2023-07-29T13:29:52.224467+0000","last_became_peered":"2023-07-29T13:29:52.224467+0000","last_unstale":"2023-07-29T14:06:18.364524+0000","last_undegraded":"2023-07-29T14:06:18.364524+0000","last_fullsized":"2023-07-29T14:06:18.364524+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:18.363719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:18.363719+0000","last_clean_scrub_stamp":"2023-07-29T14:06:18.363719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:55:50.872380+0000","scrub_duration":0.086121381999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":986,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:03.192032+0000","last_change":"2023-07-29T14:06:03.193988+0000","last_active":"2023-07-29T14:06:03.192032+0000","last_peered":"2023-07-29T14:06:03.192032+0000","last_clean":"2023-07-29T14:06:03.192032+0000","last_became_active":"2023-07-29T13:28:14.642023+0000","last_became_peered":"2023-07-29T13:28:14.642023+0000","last_unstale":"2023-07-29T14:06:03.192032+0000","last_undegraded":"2023-07-29T14:06:03.192032+0000","last_fullsized":"2023-07-29T14:06:03.192032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:03.191182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:03.191182+0000","last_clean_scrub_stamp":"2023-07-29T14:06:03.191182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:04:17.112604+0000","scrub_duration":0.035278285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":946,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:08.699663+0000","last_change":"2023-07-29T14:06:08.701416+0000","last_active":"2023-07-29T14:06:08.699663+0000","last_peered":"2023-07-29T14:06:08.699663+0000","last_clean":"2023-07-29T14:06:08.699663+0000","last_became_active":"2023-07-29T13:29:41.844314+0000","last_became_peered":"2023-07-29T13:29:41.844314+0000","last_unstale":"2023-07-29T14:06:08.699663+0000","last_undegraded":"2023-07-29T14:06:08.699663+0000","last_fullsized":"2023-07-29T14:06:08.699663+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:08.698853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:08.698853+0000","last_clean_scrub_stamp":"2023-07-29T14:06:08.698853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:09:31.454798+0000","scrub_duration":0.035659037999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":977,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:02.941253+0000","last_change":"2023-07-29T14:06:02.941253+0000","last_active":"2023-07-29T14:06:02.941253+0000","last_peered":"2023-07-29T14:06:02.941253+0000","last_clean":"2023-07-29T14:06:02.941253+0000","last_became_active":"2023-07-29T13:56:55.150514+0000","last_became_peered":"2023-07-29T13:56:55.150514+0000","last_unstale":"2023-07-29T14:06:02.941253+0000","last_undegraded":"2023-07-29T14:06:02.941253+0000","last_fullsized":"2023-07-29T14:06:02.941253+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:02.938567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:02.938567+0000","last_clean_scrub_stamp":"2023-07-29T14:06:02.938567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:27:07.832610+0000","scrub_duration":0.032938768,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1018,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:26.308197+0000","last_change":"2023-07-29T14:06:26.313316+0000","last_active":"2023-07-29T14:06:26.308197+0000","last_peered":"2023-07-29T14:06:26.308197+0000","last_clean":"2023-07-29T14:06:26.308197+0000","last_became_active":"2023-07-29T13:56:55.609414+0000","last_became_peered":"2023-07-29T13:56:55.609414+0000","last_unstale":"2023-07-29T14:06:26.308197+0000","last_undegraded":"2023-07-29T14:06:26.308197+0000","last_fullsized":"2023-07-29T14:06:26.308197+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:26.307356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:26.307356+0000","last_clean_scrub_stamp":"2023-07-29T14:06:26.307356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T23:35:31.880986+0000","scrub_duration":0.053529742999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1025,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:52.312489+0000","last_change":"2023-07-29T14:05:52.318168+0000","last_active":"2023-07-29T14:05:52.312489+0000","last_peered":"2023-07-29T14:05:52.312489+0000","last_clean":"2023-07-29T14:05:52.312489+0000","last_became_active":"2023-07-29T13:56:55.543861+0000","last_became_peered":"2023-07-29T13:56:55.543861+0000","last_unstale":"2023-07-29T14:05:52.312489+0000","last_undegraded":"2023-07-29T14:05:52.312489+0000","last_fullsized":"2023-07-29T14:05:52.312489+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:52.310408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:52.310408+0000","last_clean_scrub_stamp":"2023-07-29T14:05:52.310408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:37:08.149976+0000","scrub_duration":0.039706038999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":963,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:05.233062+0000","last_change":"2023-07-29T14:06:05.234815+0000","last_active":"2023-07-29T14:06:05.233062+0000","last_peered":"2023-07-29T14:06:05.233062+0000","last_clean":"2023-07-29T14:06:05.233062+0000","last_became_active":"2023-07-29T14:05:37.607023+0000","last_became_peered":"2023-07-29T14:05:37.607023+0000","last_unstale":"2023-07-29T14:06:05.233062+0000","last_undegraded":"2023-07-29T14:06:05.233062+0000","last_fullsized":"2023-07-29T14:06:05.233062+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:05.232236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:05.232236+0000","last_clean_scrub_stamp":"2023-07-29T14:06:05.232236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:54:00.313000+0000","scrub_duration":0.036641874999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":986,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:53.331278+0000","last_change":"2023-07-29T14:05:53.333097+0000","last_active":"2023-07-29T14:05:53.331278+0000","last_peered":"2023-07-29T14:05:53.331278+0000","last_clean":"2023-07-29T14:05:53.331278+0000","last_became_active":"2023-07-29T13:28:14.708042+0000","last_became_peered":"2023-07-29T13:28:14.708042+0000","last_unstale":"2023-07-29T14:05:53.331278+0000","last_undegraded":"2023-07-29T14:05:53.331278+0000","last_fullsized":"2023-07-29T14:05:53.331278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:53.330469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:53.330469+0000","last_clean_scrub_stamp":"2023-07-29T14:05:53.330469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:53:28.809768+0000","scrub_duration":0.040276365000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"1004'78","reported_seq":1116,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:51.159032+0000","last_change":"2023-07-29T14:05:51.166709+0000","last_active":"2023-07-29T14:05:51.159032+0000","last_peered":"2023-07-29T14:05:51.159032+0000","last_clean":"2023-07-29T14:05:51.159032+0000","last_became_active":"2023-07-29T13:56:55.157702+0000","last_became_peered":"2023-07-29T13:56:55.157702+0000","last_unstale":"2023-07-29T14:05:51.159032+0000","last_undegraded":"2023-07-29T14:05:51.159032+0000","last_fullsized":"2023-07-29T14:05:51.159032+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"1004'78","last_scrub_stamp":"2023-07-29T14:05:51.155533+0000","last_deep_scrub":"1004'78","last_deep_scrub_stamp":"2023-07-29T14:05:51.155533+0000","last_clean_scrub_stamp":"2023-07-29T14:05:51.155533+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:17:55.983069+0000","scrub_duration":0.143700576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1008,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:50.973540+0000","last_change":"2023-07-29T14:05:50.975646+0000","last_active":"2023-07-29T14:05:50.973540+0000","last_peered":"2023-07-29T14:05:50.973540+0000","last_clean":"2023-07-29T14:05:50.973540+0000","last_became_active":"2023-07-29T13:56:55.341426+0000","last_became_peered":"2023-07-29T13:56:55.341426+0000","last_unstale":"2023-07-29T14:05:50.973540+0000","last_undegraded":"2023-07-29T14:05:50.973540+0000","last_fullsized":"2023-07-29T14:05:50.973540+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:50.972716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:50.972716+0000","last_clean_scrub_stamp":"2023-07-29T14:05:50.972716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T00:38:24.126097+0000","scrub_duration":0.034645589999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":960,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:01.263330+0000","last_change":"2023-07-29T14:06:01.263330+0000","last_active":"2023-07-29T14:06:01.263330+0000","last_peered":"2023-07-29T14:06:01.263330+0000","last_clean":"2023-07-29T14:06:01.263330+0000","last_became_active":"2023-07-29T13:41:38.057649+0000","last_became_peered":"2023-07-29T13:41:38.057649+0000","last_unstale":"2023-07-29T14:06:01.263330+0000","last_undegraded":"2023-07-29T14:06:01.263330+0000","last_fullsized":"2023-07-29T14:06:01.263330+0000","mapping_epoch":447,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":448,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:01.260705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:01.260705+0000","last_clean_scrub_stamp":"2023-07-29T14:06:01.260705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:16:52.634948+0000","scrub_duration":0.065964967999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":938,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:03.967776+0000","last_change":"2023-07-29T14:06:03.973270+0000","last_active":"2023-07-29T14:06:03.967776+0000","last_peered":"2023-07-29T14:06:03.967776+0000","last_clean":"2023-07-29T14:06:03.967776+0000","last_became_active":"2023-07-29T13:53:19.560480+0000","last_became_peered":"2023-07-29T13:53:19.560480+0000","last_unstale":"2023-07-29T14:06:03.967776+0000","last_undegraded":"2023-07-29T14:06:03.967776+0000","last_fullsized":"2023-07-29T14:06:03.967776+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:03.966930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:03.966930+0000","last_clean_scrub_stamp":"2023-07-29T14:06:03.966930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T16:08:51.541717+0000","scrub_duration":0.044001921999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":926,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:10.990517+0000","last_change":"2023-07-29T14:06:10.996113+0000","last_active":"2023-07-29T14:06:10.990517+0000","last_peered":"2023-07-29T14:06:10.990517+0000","last_clean":"2023-07-29T14:06:10.990517+0000","last_became_active":"2023-07-29T13:53:19.611477+0000","last_became_peered":"2023-07-29T13:53:19.611477+0000","last_unstale":"2023-07-29T14:06:10.990517+0000","last_undegraded":"2023-07-29T14:06:10.990517+0000","last_fullsized":"2023-07-29T14:06:10.990517+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:10.989645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:10.989645+0000","last_clean_scrub_stamp":"2023-07-29T14:06:10.989645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T17:33:44.721397+0000","scrub_duration":0.077086057999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":963,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:19.074370+0000","last_change":"2023-07-29T14:06:19.076272+0000","last_active":"2023-07-29T14:06:19.074370+0000","last_peered":"2023-07-29T14:06:19.074370+0000","last_clean":"2023-07-29T14:06:19.074370+0000","last_became_active":"2023-07-29T13:56:55.173228+0000","last_became_peered":"2023-07-29T13:56:55.173228+0000","last_unstale":"2023-07-29T14:06:19.074370+0000","last_undegraded":"2023-07-29T14:06:19.074370+0000","last_fullsized":"2023-07-29T14:06:19.074370+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:19.073497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:19.073497+0000","last_clean_scrub_stamp":"2023-07-29T14:06:19.073497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:23:08.746329+0000","scrub_duration":0.032057964000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":951,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:23.207310+0000","last_change":"2023-07-29T14:06:23.221024+0000","last_active":"2023-07-29T14:06:23.207310+0000","last_peered":"2023-07-29T14:06:23.207310+0000","last_clean":"2023-07-29T14:06:23.207310+0000","last_became_active":"2023-07-29T13:56:55.238036+0000","last_became_peered":"2023-07-29T13:56:55.238036+0000","last_unstale":"2023-07-29T14:06:23.207310+0000","last_undegraded":"2023-07-29T14:06:23.207310+0000","last_fullsized":"2023-07-29T14:06:23.207310+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:23.206482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:23.206482+0000","last_clean_scrub_stamp":"2023-07-29T14:06:23.206482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T14:54:38.280521+0000","scrub_duration":0.037775228000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":945,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:11.802047+0000","last_change":"2023-07-29T14:06:11.802047+0000","last_active":"2023-07-29T14:06:11.802047+0000","last_peered":"2023-07-29T14:06:11.802047+0000","last_clean":"2023-07-29T14:06:11.802047+0000","last_became_active":"2023-07-29T13:29:33.508113+0000","last_became_peered":"2023-07-29T13:29:33.508113+0000","last_unstale":"2023-07-29T14:06:11.802047+0000","last_undegraded":"2023-07-29T14:06:11.802047+0000","last_fullsized":"2023-07-29T14:06:11.802047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:11.795786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:11.795786+0000","last_clean_scrub_stamp":"2023-07-29T14:06:11.795786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T18:00:12.237098+0000","scrub_duration":0.046529005999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":953,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:05:55.179499+0000","last_change":"2023-07-29T14:05:55.185047+0000","last_active":"2023-07-29T14:05:55.179499+0000","last_peered":"2023-07-29T14:05:55.179499+0000","last_clean":"2023-07-29T14:05:55.179499+0000","last_became_active":"2023-07-29T13:29:32.123788+0000","last_became_peered":"2023-07-29T13:29:32.123788+0000","last_unstale":"2023-07-29T14:05:55.179499+0000","last_undegraded":"2023-07-29T14:05:55.179499+0000","last_fullsized":"2023-07-29T14:05:55.179499+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:05:55.178683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:05:55.178683+0000","last_clean_scrub_stamp":"2023-07-29T14:05:55.178683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:10:55.395109+0000","scrub_duration":0.070950845999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":950,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:11.152661+0000","last_change":"2023-07-29T14:06:11.152661+0000","last_active":"2023-07-29T14:06:11.152661+0000","last_peered":"2023-07-29T14:06:11.152661+0000","last_clean":"2023-07-29T14:06:11.152661+0000","last_became_active":"2023-07-29T13:56:55.591631+0000","last_became_peered":"2023-07-29T13:56:55.591631+0000","last_unstale":"2023-07-29T14:06:11.152661+0000","last_undegraded":"2023-07-29T14:06:11.152661+0000","last_fullsized":"2023-07-29T14:06:11.152661+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:11.146780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:11.146780+0000","last_clean_scrub_stamp":"2023-07-29T14:06:11.146780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T20:07:27.329639+0000","scrub_duration":0.031543092000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1016,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:10.187646+0000","last_change":"2023-07-29T14:06:10.190321+0000","last_active":"2023-07-29T14:06:10.187646+0000","last_peered":"2023-07-29T14:06:10.187646+0000","last_clean":"2023-07-29T14:06:10.187646+0000","last_became_active":"2023-07-29T13:56:55.564155+0000","last_became_peered":"2023-07-29T13:56:55.564155+0000","last_unstale":"2023-07-29T14:06:10.187646+0000","last_undegraded":"2023-07-29T14:06:10.187646+0000","last_fullsized":"2023-07-29T14:06:10.187646+0000","mapping_epoch":704,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":705,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:10.186791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:10.186791+0000","last_clean_scrub_stamp":"2023-07-29T14:06:10.186791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-31T01:01:01.099504+0000","scrub_duration":0.066176493000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":986,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:00.245088+0000","last_change":"2023-07-29T14:06:00.247048+0000","last_active":"2023-07-29T14:06:00.245088+0000","last_peered":"2023-07-29T14:06:00.245088+0000","last_clean":"2023-07-29T14:06:00.245088+0000","last_became_active":"2023-07-29T13:53:19.433950+0000","last_became_peered":"2023-07-29T13:53:19.433950+0000","last_unstale":"2023-07-29T14:06:00.245088+0000","last_undegraded":"2023-07-29T14:06:00.245088+0000","last_fullsized":"2023-07-29T14:06:00.245088+0000","mapping_epoch":650,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":651,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:00.244224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:00.244224+0000","last_clean_scrub_stamp":"2023-07-29T14:06:00.244224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:56:06.799170+0000","scrub_duration":0.044199228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":972,"reported_epoch":1004,"state":"active+clean","last_fresh":"2023-07-29T14:06:03.254844+0000","last_change":"2023-07-29T14:06:03.259347+0000","last_active":"2023-07-29T14:06:03.254844+0000","last_peered":"2023-07-29T14:06:03.254844+0000","last_clean":"2023-07-29T14:06:03.254844+0000","last_became_active":"2023-07-29T14:05:37.513918+0000","last_became_peered":"2023-07-29T14:05:37.513918+0000","last_unstale":"2023-07-29T14:06:03.254844+0000","last_undegraded":"2023-07-29T14:06:03.254844+0000","last_fullsized":"2023-07-29T14:06:03.254844+0000","mapping_epoch":1002,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":1003,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-29T14:06:03.254014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-29T14:06:03.254014+0000","last_clean_scrub_stamp":"2023-07-29T14:06:03.254014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-30T21:25:43.226591+0000","scrub_duration":0.081281532000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":17,"seq":73014444483,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59436,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312404,"statfs":{"total":96636764160,"available":96575901696,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":15.343999999999999,"5min":16.634,"15min":170.43700000000001},"min":{"1min":6.3490000000000002,"5min":6.3330000000000002,"15min":4.7590000000000003},"max":{"1min":111.66,"5min":172.44399999999999,"15min":12751.485000000001},"last":8.6560000000000006},{"interface":"front","average":{"1min":16.254000000000001,"5min":16.774000000000001,"15min":173.22},"min":{"1min":6.79,"5min":5.4500000000000002,"15min":5.4500000000000002},"max":{"1min":129.28399999999999,"5min":173.48699999999999,"15min":12747.76},"last":12.164999999999999}]},{"osd":1,"last update":"Sat Jul 29 14:05:52 2023","interfaces":[{"interface":"back","average":{"1min":16.469999999999999,"5min":1542.5060000000001,"15min":525.53399999999999},"min":{"1min":6.0339999999999998,"5min":6.0339999999999998,"15min":6.0339999999999998},"max":{"1min":88.435000000000002,"5min":29038.089,"15min":29038.089},"last":15.5},{"interface":"front","average":{"1min":22.672999999999998,"5min":1548.559,"15min":529.87699999999995},"min":{"1min":6.0449999999999999,"5min":6.0449999999999999,"15min":6.0449999999999999},"max":{"1min":180.447,"5min":29039.032999999999,"15min":29039.032999999999},"last":11.042999999999999}]},{"osd":2,"last update":"Sat Jul 29 14:05:52 2023","interfaces":[{"interface":"back","average":{"1min":29.186,"5min":23.222999999999999,"15min":21.166},"min":{"1min":7.1050000000000004,"5min":7.1050000000000004,"15min":7.1050000000000004},"max":{"1min":200.35400000000001,"5min":249.84800000000001,"15min":784.66200000000003},"last":7.8109999999999999},{"interface":"front","average":{"1min":27.800000000000001,"5min":22.724,"15min":19.367999999999999},"min":{"1min":7.6959999999999997,"5min":7.6959999999999997,"15min":7.6959999999999997},"max":{"1min":191.994,"5min":202.64500000000001,"15min":368.62200000000001},"last":18.562000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:05:56 2023","interfaces":[{"interface":"back","average":{"1min":15.656000000000001,"5min":25.140999999999998,"15min":478.09899999999999},"min":{"1min":7.4889999999999999,"5min":6.4740000000000002,"15min":6.4740000000000002},"max":{"1min":63.453000000000003,"5min":312.73599999999999,"15min":28381.707999999999},"last":9.5549999999999997},{"interface":"front","average":{"1min":23.655999999999999,"5min":28.308,"15min":473.64299999999997},"min":{"1min":10.977,"5min":9.3219999999999992,"15min":9.2739999999999991},"max":{"1min":137.84299999999999,"5min":242.63,"15min":28126.012999999999},"last":16.574000000000002}]},{"osd":5,"last update":"Sat Jul 29 14:06:06 2023","interfaces":[{"interface":"back","average":{"1min":35.478000000000002,"5min":22.574999999999999,"15min":32.390999999999998},"min":{"1min":10.343,"5min":8.4540000000000006,"15min":7.3689999999999998},"max":{"1min":216.00299999999999,"5min":216.00299999999999,"15min":488.72300000000001},"last":20.959},{"interface":"front","average":{"1min":35.033999999999999,"5min":21.722000000000001,"15min":22.565000000000001},"min":{"1min":10.356999999999999,"5min":8.1379999999999999,"15min":7.3840000000000003},"max":{"1min":212.72999999999999,"5min":212.72999999999999,"15min":212.72999999999999},"last":17.199000000000002}]},{"osd":6,"last update":"Sat Jul 29 14:05:52 2023","interfaces":[{"interface":"back","average":{"1min":25.501999999999999,"5min":20.158999999999999,"15min":22.632999999999999},"min":{"1min":9.1099999999999994,"5min":8.0129999999999999,"15min":7.7779999999999996},"max":{"1min":203.011,"5min":203.011,"15min":787.18499999999995},"last":10.473000000000001},{"interface":"front","average":{"1min":27.638999999999999,"5min":21.126000000000001,"15min":22.797000000000001},"min":{"1min":9.6300000000000008,"5min":8.5790000000000006,"15min":8.3170000000000002},"max":{"1min":205.321,"5min":205.321,"15min":646.87599999999998},"last":13.474}]},{"osd":7,"last update":"Sat Jul 29 14:05:52 2023","interfaces":[{"interface":"back","average":{"1min":32.484999999999999,"5min":23.815000000000001,"15min":44.118000000000002},"min":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"max":{"1min":208.702,"5min":208.702,"15min":648.54200000000003},"last":19.763000000000002},{"interface":"front","average":{"1min":24.163,"5min":20.629000000000001,"15min":34.918999999999997},"min":{"1min":8.6259999999999994,"5min":8.6259999999999994,"15min":8.6259999999999994},"max":{"1min":108.06399999999999,"5min":108.06399999999999,"15min":364.86500000000001},"last":14.103}]}]},{"osd":2,"up_from":17,"seq":73014444495,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63536,"kb_used_data":6248,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94308304,"statfs":{"total":96636764160,"available":96571703296,"internally_reserved":0,"allocated":6397952,"data_stored":3446913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":13.247999999999999,"15min":130.82599999999999},"min":{"1min":5.3789999999999996,"5min":5.0209999999999999,"15min":4.476},"max":{"1min":29.634,"5min":93.590999999999994,"15min":10009.493},"last":14.193},{"interface":"front","average":{"1min":13.936999999999999,"5min":13.526,"15min":131.76599999999999},"min":{"1min":7.1299999999999999,"5min":6.2069999999999999,"15min":5.617},"max":{"1min":37.527999999999999,"5min":99.156999999999996,"15min":9995.5869999999995},"last":8.2739999999999991}]},{"osd":1,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":16.233000000000001,"5min":1412.377,"15min":481.59100000000001},"min":{"1min":6.7910000000000004,"5min":6.0899999999999999,"15min":6.0899999999999999},"max":{"1min":101.836,"5min":29343.018,"15min":29343.018},"last":7.6260000000000003},{"interface":"front","average":{"1min":16.434000000000001,"5min":1411.5540000000001,"15min":479.24299999999999},"min":{"1min":5.7990000000000004,"5min":5.7990000000000004,"15min":5.7990000000000004},"max":{"1min":137.94,"5min":29342.263999999999,"15min":29342.263999999999},"last":10.305}]},{"osd":3,"last update":"Sat Jul 29 14:05:44 2023","interfaces":[{"interface":"back","average":{"1min":13.795,"5min":19.425999999999998,"15min":365.00299999999999},"min":{"1min":8.1449999999999996,"5min":6.7809999999999997,"15min":6.7809999999999997},"max":{"1min":30.794,"5min":331.92399999999998,"15min":29770.313999999998},"last":9.5820000000000007},{"interface":"front","average":{"1min":13.835000000000001,"5min":21.202000000000002,"15min":358.64400000000001},"min":{"1min":7.351,"5min":7.351,"15min":6.4749999999999996},"max":{"1min":34.841000000000001,"5min":332.49900000000002,"15min":29586.387999999999},"last":8.8930000000000007}]},{"osd":4,"last update":"Sat Jul 29 14:05:53 2023","interfaces":[{"interface":"back","average":{"1min":14.763,"5min":814.56100000000004,"15min":870.06399999999996},"min":{"1min":6.5810000000000004,"5min":6.5810000000000004,"15min":6.5810000000000004},"max":{"1min":80.256,"5min":25685.629000000001,"15min":32071.659},"last":15.946},{"interface":"front","average":{"1min":15.894,"5min":810.43799999999999,"15min":871.18799999999999},"min":{"1min":8.0410000000000004,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":52.790999999999997,"5min":25610.863000000001,"15min":32126.150000000001},"last":12.321}]},{"osd":5,"last update":"Sat Jul 29 14:06:11 2023","interfaces":[{"interface":"back","average":{"1min":15.445,"5min":18.456,"15min":35.558999999999997},"min":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":35.451000000000001,"5min":193.45400000000001,"15min":428.04000000000002},"last":11.460000000000001},{"interface":"front","average":{"1min":18.681000000000001,"5min":19.016999999999999,"15min":38.996000000000002},"min":{"1min":7.2960000000000003,"5min":6.891,"15min":6.891},"max":{"1min":77.212999999999994,"5min":153.148,"15min":430.15499999999997},"last":16.529}]},{"osd":6,"last update":"Sat Jul 29 14:05:40 2023","interfaces":[{"interface":"back","average":{"1min":17.140000000000001,"5min":21.350999999999999,"15min":20.024000000000001},"min":{"1min":8.3450000000000006,"5min":8.3450000000000006,"15min":8.3450000000000006},"max":{"1min":77.855000000000004,"5min":318.45299999999997,"15min":318.45299999999997},"last":10.903},{"interface":"front","average":{"1min":17.119,"5min":21.058,"15min":20.518999999999998},"min":{"1min":9.2230000000000008,"5min":9.2230000000000008,"15min":7.5300000000000002},"max":{"1min":44.755000000000003,"5min":349.34899999999999,"15min":349.34899999999999},"last":15.356}]},{"osd":7,"last update":"Sat Jul 29 14:05:44 2023","interfaces":[{"interface":"back","average":{"1min":19.048999999999999,"5min":19.532,"15min":21.023},"min":{"1min":10.064,"5min":8.5079999999999991,"15min":7.5890000000000004},"max":{"1min":80.870000000000005,"5min":291.80900000000003,"15min":291.80900000000003},"last":14.797000000000001},{"interface":"front","average":{"1min":22.766999999999999,"5min":21.436,"15min":22.666},"min":{"1min":9.3379999999999992,"5min":9.1940000000000008,"15min":9.1940000000000008},"max":{"1min":82.373999999999995,"5min":291.44200000000001,"15min":291.44200000000001},"last":17.123999999999999}]}]},{"osd":1,"up_from":447,"seq":1919850381753,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59832,"kb_used_data":6640,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312008,"statfs":{"total":96636764160,"available":96575496192,"internally_reserved":0,"allocated":6799360,"data_stored":3844753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":20.931000000000001,"5min":20.015999999999998,"15min":112.807},"min":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.3680000000000003},"max":{"1min":116.512,"5min":364.47800000000001,"15min":12525.57},"last":9.1959999999999997},{"interface":"front","average":{"1min":18.943000000000001,"5min":20.664000000000001,"15min":112.874},"min":{"1min":6.0970000000000004,"5min":6.0970000000000004,"15min":4.6820000000000004},"max":{"1min":100.402,"5min":372.00999999999999,"15min":12523.974},"last":8.3339999999999996}]},{"osd":2,"last update":"Sat Jul 29 14:05:55 2023","interfaces":[{"interface":"back","average":{"1min":12.737,"5min":24.844999999999999,"15min":19.154},"min":{"1min":5.6580000000000004,"5min":5.6580000000000004,"15min":5.5919999999999996},"max":{"1min":40.566000000000003,"5min":360.88099999999997,"15min":360.88099999999997},"last":9.8179999999999996},{"interface":"front","average":{"1min":13.837999999999999,"5min":25.808,"15min":20.196000000000002},"min":{"1min":6.6120000000000001,"5min":6.6120000000000001,"15min":6.2729999999999997},"max":{"1min":59.558999999999997,"5min":368.51400000000001,"15min":368.51400000000001},"last":13.355}]},{"osd":3,"last update":"Sat Jul 29 14:05:55 2023","interfaces":[{"interface":"back","average":{"1min":12.858000000000001,"5min":27.369,"15min":531.38300000000004},"min":{"1min":8.468,"5min":6.7140000000000004,"15min":6.7140000000000004},"max":{"1min":21.891999999999999,"5min":435.92200000000003,"15min":34504.942000000003},"last":11.742000000000001},{"interface":"front","average":{"1min":18.202999999999999,"5min":38.177999999999997,"15min":540.05899999999997},"min":{"1min":10.888,"5min":10.423,"15min":5.3639999999999999},"max":{"1min":37.070999999999998,"5min":700.10500000000002,"15min":34620.970999999998},"last":16.684000000000001}]},{"osd":4,"last update":"Sat Jul 29 14:05:55 2023","interfaces":[{"interface":"back","average":{"1min":12.792,"5min":1168.114,"15min":980.44200000000001},"min":{"1min":7.9939999999999998,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":19.539999999999999,"5min":25247.098999999998,"15min":32258.529999999999},"last":15.933999999999999},{"interface":"front","average":{"1min":15.993,"5min":1162.9349999999999,"15min":967.01300000000003},"min":{"1min":9.3900000000000006,"5min":9.3900000000000006,"15min":6.5090000000000003},"max":{"1min":35.779000000000003,"5min":25174.008999999998,"15min":31933.992999999999},"last":17.443999999999999}]},{"osd":5,"last update":"Sat Jul 29 14:06:15 2023","interfaces":[{"interface":"back","average":{"1min":19.119,"5min":33.439999999999998,"15min":41.228000000000002},"min":{"1min":7.7309999999999999,"5min":7.7309999999999999,"15min":7.7309999999999999},"max":{"1min":81.477999999999994,"5min":433.79300000000001,"15min":433.79300000000001},"last":14.162000000000001},{"interface":"front","average":{"1min":24.742000000000001,"5min":36.497,"15min":44.088999999999999},"min":{"1min":13.519,"5min":9.6929999999999996,"15min":8.7260000000000009},"max":{"1min":88.980999999999995,"5min":419.22800000000001,"15min":419.22800000000001},"last":18.199000000000002}]},{"osd":6,"last update":"Sat Jul 29 14:05:55 2023","interfaces":[{"interface":"back","average":{"1min":10.282999999999999,"5min":20.245999999999999,"15min":18.065999999999999},"min":{"1min":6.8520000000000003,"5min":6.6079999999999997,"15min":6.327},"max":{"1min":17.503,"5min":357.20499999999998,"15min":357.20499999999998},"last":8.6370000000000005},{"interface":"front","average":{"1min":13.545,"5min":25.114999999999998,"15min":22.295000000000002},"min":{"1min":7.9039999999999999,"5min":7.9039999999999999,"15min":7.9039999999999999},"max":{"1min":19.331,"5min":359.68099999999998,"15min":359.68099999999998},"last":11.023}]},{"osd":7,"last update":"Sat Jul 29 14:05:55 2023","interfaces":[{"interface":"back","average":{"1min":16.536999999999999,"5min":33.811,"15min":23.893999999999998},"min":{"1min":11.493,"5min":8.5199999999999996,"15min":7.5919999999999996},"max":{"1min":24.332999999999998,"5min":563.29700000000003,"15min":563.29700000000003},"last":14.718999999999999},{"interface":"front","average":{"1min":15.179,"5min":29.501000000000001,"15min":21.321999999999999},"min":{"1min":9.4250000000000007,"5min":8.2240000000000002,"15min":7.2919999999999998},"max":{"1min":35.567999999999998,"5min":352.65199999999999,"15min":352.65199999999999},"last":10.471}]}]},{"osd":0,"up_from":1002,"seq":4303557231039,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59436,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312404,"statfs":{"total":96636764160,"available":96575901696,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":71.099999999999994,"5min":1023.201,"15min":351.10899999999998},"min":{"1min":8.9689999999999994,"5min":5.9249999999999998,"15min":4.6269999999999998},"max":{"1min":350.00200000000001,"5min":27534,"15min":27534},"last":11.494999999999999},{"interface":"front","average":{"1min":99.875,"5min":1025.7080000000001,"15min":350.98899999999998},"min":{"1min":5.9589999999999996,"5min":5.7240000000000002,"15min":5.0739999999999998},"max":{"1min":591.69600000000003,"5min":27535.008999999998,"15min":27535.008999999998},"last":12.686}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.170999999999999,"5min":22.285,"15min":18.638000000000002},"min":{"1min":11.680999999999999,"5min":8.1069999999999993,"15min":6.6159999999999997},"max":{"1min":364.62799999999999,"5min":364.62799999999999,"15min":364.62799999999999},"last":9.5879999999999992},{"interface":"front","average":{"1min":45.890000000000001,"5min":26.417999999999999,"15min":21.443000000000001},"min":{"1min":14.172000000000001,"5min":10.023999999999999,"15min":8.8780000000000001},"max":{"1min":370.34300000000002,"5min":370.34300000000002,"15min":370.34300000000002},"last":12.087}]},{"osd":3,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":66.832999999999998,"5min":25.178000000000001,"15min":355.89100000000002},"min":{"1min":6.7610000000000001,"5min":6.4409999999999998,"15min":5.891},"max":{"1min":444.59500000000003,"5min":444.59500000000003,"15min":30170.040000000001},"last":7.0359999999999996},{"interface":"front","average":{"1min":65.754000000000005,"5min":33.606000000000002,"15min":363.07400000000001},"min":{"1min":10.266999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"max":{"1min":386.06099999999998,"5min":386.06099999999998,"15min":30171.194},"last":10.288}]},{"osd":4,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":92.555000000000007,"5min":36.014000000000003,"15min":585.43899999999996},"min":{"1min":12.201000000000001,"5min":6.4480000000000004,"15min":6.0979999999999999},"max":{"1min":445.65300000000002,"5min":445.65300000000002,"15min":29892.669999999998},"last":13.403},{"interface":"front","average":{"1min":115.806,"5min":40.560000000000002,"15min":588.88900000000001},"min":{"1min":7.923,"5min":5.9400000000000004,"15min":5.9400000000000004},"max":{"1min":593.13499999999999,"5min":593.13499999999999,"15min":29877.289000000001},"last":8.5150000000000006}]},{"osd":5,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":40.341000000000001,"5min":28.972000000000001,"15min":56.932000000000002},"min":{"1min":12.702,"5min":9.5700000000000003,"15min":9.5700000000000003},"max":{"1min":403.13999999999999,"5min":403.13999999999999,"15min":508.02100000000002},"last":17.613},{"interface":"front","average":{"1min":43.966999999999999,"5min":23.216000000000001,"15min":32.374000000000002},"min":{"1min":7.6429999999999998,"5min":7.1210000000000004,"15min":7.1210000000000004},"max":{"1min":590.58500000000004,"5min":590.58500000000004,"15min":590.58500000000004},"last":16.597000000000001}]},{"osd":6,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":52.365000000000002,"5min":30.853999999999999,"15min":25.227},"min":{"1min":16.876000000000001,"5min":10.409000000000001,"15min":9.9350000000000005},"max":{"1min":375.36500000000001,"5min":375.36500000000001,"15min":375.36500000000001},"last":10.914},{"interface":"front","average":{"1min":55.634,"5min":32.579999999999998,"15min":27.417999999999999},"min":{"1min":18.984999999999999,"5min":11.564,"15min":11.564},"max":{"1min":380.31599999999997,"5min":380.31599999999997,"15min":380.31599999999997},"last":14.006}]},{"osd":7,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":57.960999999999999,"5min":28.638000000000002,"15min":20.780000000000001},"min":{"1min":17.748999999999999,"5min":8.4939999999999998,"15min":7.9349999999999996},"max":{"1min":385.01900000000001,"5min":385.01900000000001,"15min":385.01900000000001},"last":14.616},{"interface":"front","average":{"1min":54.853999999999999,"5min":29.765999999999998,"15min":25.545000000000002},"min":{"1min":10.992000000000001,"5min":8.1229999999999993,"15min":8.1229999999999993},"max":{"1min":446.69099999999997,"5min":446.69099999999997,"15min":446.69099999999997},"last":18.416}]}]},{"osd":4,"up_from":16,"seq":68719477172,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59436,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312404,"statfs":{"total":96636764160,"available":96575901696,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:51 2023","interfaces":[{"interface":"back","average":{"1min":13.432,"5min":21.457000000000001,"15min":87.253},"min":{"1min":6.4029999999999996,"5min":4.6230000000000002,"15min":4.6230000000000002},"max":{"1min":48.048999999999999,"5min":495.86900000000003,"15min":10627.831},"last":7.609},{"interface":"front","average":{"1min":18.010000000000002,"5min":25.013000000000002,"15min":90.852000000000004},"min":{"1min":7.2400000000000002,"5min":6.0990000000000002,"15min":5.6660000000000004},"max":{"1min":124.64100000000001,"5min":446.84100000000001,"15min":10653.877},"last":17.315999999999999}]},{"osd":1,"last update":"Sat Jul 29 14:05:54 2023","interfaces":[{"interface":"back","average":{"1min":11.929,"5min":1441.175,"15min":495.887},"min":{"1min":7.21,"5min":7.21,"15min":6.7960000000000003},"max":{"1min":15.911,"5min":26872.857,"15min":26872.857},"last":12.561},{"interface":"front","average":{"1min":12.18,"5min":1440.078,"15min":493.22500000000002},"min":{"1min":7.5720000000000001,"5min":7.4000000000000004,"15min":5.6909999999999998},"max":{"1min":19.245999999999999,"5min":26874.048999999999,"15min":26874.048999999999},"last":10.922000000000001}]},{"osd":2,"last update":"Sat Jul 29 14:06:30 2023","interfaces":[{"interface":"back","average":{"1min":13.101000000000001,"5min":21.297000000000001,"15min":23.056000000000001},"min":{"1min":8.5600000000000005,"5min":7.3019999999999996,"15min":7.3019999999999996},"max":{"1min":17.305,"5min":453.40100000000001,"15min":597.11800000000005},"last":16.692},{"interface":"front","average":{"1min":17.25,"5min":24.184999999999999,"15min":26.364999999999998},"min":{"1min":7.7919999999999998,"5min":7.7919999999999998,"15min":7.7919999999999998},"max":{"1min":57.628,"5min":453.96199999999999,"15min":705.04499999999996},"last":20.405999999999999}]},{"osd":3,"last update":"Sat Jul 29 14:05:54 2023","interfaces":[{"interface":"back","average":{"1min":16.591000000000001,"5min":27.045999999999999,"15min":590.73400000000004},"min":{"1min":13.207000000000001,"5min":9.2539999999999996,"15min":5.9740000000000002},"max":{"1min":43.826000000000001,"5min":448.65100000000001,"15min":30725.612000000001},"last":19.460000000000001},{"interface":"front","average":{"1min":14.129,"5min":20.515999999999998,"15min":580.85900000000004},"min":{"1min":6.641,"5min":6.3840000000000003,"15min":6.3840000000000003},"max":{"1min":45.087000000000003,"5min":336.62599999999998,"15min":30407.659},"last":9.2100000000000009}]},{"osd":5,"last update":"Sat Jul 29 14:06:06 2023","interfaces":[{"interface":"back","average":{"1min":19.056000000000001,"5min":28.420000000000002,"15min":31.863},"min":{"1min":11.228999999999999,"5min":7.1440000000000001,"15min":7.1440000000000001},"max":{"1min":58.966999999999999,"5min":450.81700000000001,"15min":450.81700000000001},"last":22.492999999999999},{"interface":"front","average":{"1min":13.9,"5min":19.434999999999999,"15min":22.106000000000002},"min":{"1min":7.3419999999999996,"5min":6.2779999999999996,"15min":6.2779999999999996},"max":{"1min":63.125,"5min":340.04000000000002,"15min":340.04000000000002},"last":11.298}]},{"osd":6,"last update":"Sat Jul 29 14:06:30 2023","interfaces":[{"interface":"back","average":{"1min":13.738,"5min":20.382000000000001,"15min":22.809000000000001},"min":{"1min":9.5969999999999995,"5min":7.1689999999999996,"15min":6.1520000000000001},"max":{"1min":20.384,"5min":339.47199999999998,"15min":696.952},"last":17.98},{"interface":"front","average":{"1min":14.529,"5min":22.469000000000001,"15min":23.422000000000001},"min":{"1min":9.2200000000000006,"5min":7.1859999999999999,"15min":6.3449999999999998},"max":{"1min":21.033000000000001,"5min":498.26999999999998,"15min":598.86900000000003},"last":8.5809999999999995}]},{"osd":7,"last update":"Sat Jul 29 14:05:54 2023","interfaces":[{"interface":"back","average":{"1min":15.851000000000001,"5min":21.326000000000001,"15min":24.811},"min":{"1min":7.7300000000000004,"5min":7.7300000000000004,"15min":7.7300000000000004},"max":{"1min":20.425999999999998,"5min":340.61900000000003,"15min":699.31799999999998},"last":23.643999999999998},{"interface":"front","average":{"1min":15.250999999999999,"5min":28.420999999999999,"15min":32.200000000000003},"min":{"1min":8.7230000000000008,"5min":8.7230000000000008,"15min":8.7230000000000008},"max":{"1min":21.388999999999999,"5min":455.339,"15min":703.68200000000002},"last":16.289999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81580,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94290260,"statfs":{"total":96636764160,"available":96553226240,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":12.891999999999999,"5min":16.984000000000002,"15min":62.423000000000002},"min":{"1min":6.9390000000000001,"5min":5.2350000000000003,"15min":5.2350000000000003},"max":{"1min":29.821000000000002,"5min":224.93899999999999,"15min":9502.5869999999995},"last":13.939},{"interface":"front","average":{"1min":15.925000000000001,"5min":19.867000000000001,"15min":65.921999999999997},"min":{"1min":8.2050000000000001,"5min":6.1580000000000004,"15min":6.1580000000000004},"max":{"1min":31.303999999999998,"5min":130.48400000000001,"15min":9511.3719999999994},"last":11.728999999999999}]},{"osd":1,"last update":"Sat Jul 29 14:05:51 2023","interfaces":[{"interface":"back","average":{"1min":11.551,"5min":969.16300000000001,"15min":335.10300000000001},"min":{"1min":7.1070000000000002,"5min":7.1070000000000002,"15min":4.8529999999999998},"max":{"1min":15.645,"5min":26903.921999999999,"15min":26903.921999999999},"last":9.923},{"interface":"front","average":{"1min":10.327999999999999,"5min":963.16499999999996,"15min":331.75799999999998},"min":{"1min":5.3620000000000001,"5min":5.3620000000000001,"15min":5.3330000000000002},"max":{"1min":17.975999999999999,"5min":26893.022000000001,"15min":26893.022000000001},"last":10.82}]},{"osd":2,"last update":"Sat Jul 29 14:05:33 2023","interfaces":[{"interface":"back","average":{"1min":14.891999999999999,"5min":21.295999999999999,"15min":19.027000000000001},"min":{"1min":10.173,"5min":7.8609999999999998,"15min":6.6299999999999999},"max":{"1min":26.302,"5min":191.73099999999999,"15min":191.73099999999999},"last":8.6400000000000006},{"interface":"front","average":{"1min":11.848000000000001,"5min":16.003,"15min":15.422000000000001},"min":{"1min":7.4690000000000003,"5min":5.5499999999999998,"15min":5.5499999999999998},"max":{"1min":18,"5min":189.40199999999999,"15min":189.40199999999999},"last":15.103999999999999}]},{"osd":3,"last update":"Sat Jul 29 14:05:46 2023","interfaces":[{"interface":"back","average":{"1min":11.444000000000001,"5min":15.523999999999999,"15min":521.29999999999995},"min":{"1min":7.2489999999999997,"5min":6.335,"15min":6.335},"max":{"1min":37.712000000000003,"5min":187.852,"15min":30710.619999999999},"last":9.0839999999999996},{"interface":"front","average":{"1min":15.505000000000001,"5min":18.006,"15min":520.84199999999998},"min":{"1min":10.118,"5min":8.8770000000000007,"15min":7.6180000000000003},"max":{"1min":38.539999999999999,"5min":101.88200000000001,"15min":30591.776999999998},"last":18.195}]},{"osd":4,"last update":"Sat Jul 29 14:05:56 2023","interfaces":[{"interface":"back","average":{"1min":17.323,"5min":807.66700000000003,"15min":1018.088},"min":{"1min":8.2870000000000008,"5min":6.718,"15min":6.1070000000000002},"max":{"1min":70.116,"5min":25620.438999999998,"15min":32899.582000000002},"last":16.812999999999999},{"interface":"front","average":{"1min":17.379999999999999,"5min":826.65300000000002,"15min":1008.728},"min":{"1min":9.407,"5min":6.2089999999999996,"15min":6.2089999999999996},"max":{"1min":74.122,"5min":25979.712,"15min":32485.278999999999},"last":15.877000000000001}]},{"osd":5,"last update":"Sat Jul 29 14:06:08 2023","interfaces":[{"interface":"back","average":{"1min":21.344999999999999,"5min":21.481000000000002,"15min":24.541},"min":{"1min":10.964,"5min":10.029,"15min":8.9290000000000003},"max":{"1min":94.355000000000004,"5min":94.355000000000004,"15min":189.423},"last":12.6},{"interface":"front","average":{"1min":22.614999999999998,"5min":18.805,"15min":21.620000000000001},"min":{"1min":10.656000000000001,"5min":7.3330000000000002,"15min":7.3330000000000002},"max":{"1min":91.700999999999993,"5min":91.700999999999993,"15min":188.518},"last":17.442}]},{"osd":7,"last update":"Sat Jul 29 14:05:46 2023","interfaces":[{"interface":"back","average":{"1min":15.869,"5min":19.315000000000001,"15min":19.748999999999999},"min":{"1min":12.207000000000001,"5min":7.1059999999999999,"15min":7.1059999999999999},"max":{"1min":19.248999999999999,"5min":74.337000000000003,"15min":200.55099999999999},"last":13.226000000000001},{"interface":"front","average":{"1min":14.26,"5min":19.501999999999999,"15min":21.978000000000002},"min":{"1min":9.8089999999999993,"5min":8.0790000000000006,"15min":8.0790000000000006},"max":{"1min":19.190999999999999,"5min":89.125,"15min":222.65199999999999},"last":18.829000000000001}]}]},{"osd":5,"up_from":704,"seq":3023656976783,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77876,"kb_used_data":6636,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94293964,"statfs":{"total":96636764160,"available":96557019136,"internally_reserved":0,"allocated":6795264,"data_stored":3844748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:04:00 2023","interfaces":[{"interface":"back","average":{"1min":23.48,"5min":22.623000000000001,"15min":41.000999999999998},"min":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":4.742},"max":{"1min":150.63,"5min":262.56099999999998,"15min":299.87799999999999},"last":11.068},{"interface":"front","average":{"1min":24.837,"5min":22.727,"15min":52.545999999999999},"min":{"1min":5.8259999999999996,"5min":4.6539999999999999,"15min":4.6539999999999999},"max":{"1min":111.14100000000001,"5min":253.80000000000001,"15min":370.00999999999999},"last":7.8620000000000001}]},{"osd":1,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":10.637,"5min":22.524000000000001,"15min":31.143000000000001},"min":{"1min":7.0529999999999999,"5min":5.3780000000000001,"15min":5.3780000000000001},"max":{"1min":18.199999999999999,"5min":362.58800000000002,"15min":362.58800000000002},"last":9.3360000000000003},{"interface":"front","average":{"1min":12.446,"5min":30.510999999999999,"15min":52.337000000000003},"min":{"1min":7.0510000000000002,"5min":6.8680000000000003,"15min":6.8680000000000003},"max":{"1min":23.666,"5min":356.69900000000001,"15min":363.50999999999999},"last":7.2300000000000004}]},{"osd":2,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":13.893000000000001,"5min":21.827000000000002,"15min":47.597000000000001},"min":{"1min":6.0259999999999998,"5min":6.0259999999999998,"15min":6.0259999999999998},"max":{"1min":25.013999999999999,"5min":312.755,"15min":384.024},"last":12.444000000000001},{"interface":"front","average":{"1min":14.029999999999999,"5min":16.576000000000001,"15min":24.143999999999998},"min":{"1min":8.4190000000000005,"5min":7.2759999999999998,"15min":7.2709999999999999},"max":{"1min":22.393999999999998,"5min":210.506,"15min":228.55600000000001},"last":11.696}]},{"osd":3,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":12.836,"5min":25.149000000000001,"15min":51.404000000000003},"min":{"1min":7.4240000000000004,"5min":7.4100000000000001,"15min":7.4100000000000001},"max":{"1min":19.074999999999999,"5min":312.12200000000001,"15min":381.23200000000003},"last":13.097},{"interface":"front","average":{"1min":14.856999999999999,"5min":23.690000000000001,"15min":61.159999999999997},"min":{"1min":8.9109999999999996,"5min":7.944,"15min":7.944},"max":{"1min":24.484999999999999,"5min":263.30200000000002,"15min":382.779},"last":14.339}]},{"osd":4,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":17.190000000000001,"5min":1106.981,"15min":408.33800000000002},"min":{"1min":8.423,"5min":7.343,"15min":7.3070000000000004},"max":{"1min":42.877000000000002,"5min":23572.741999999998,"15min":23572.741999999998},"last":15.936999999999999},{"interface":"front","average":{"1min":14.066000000000001,"5min":1111.7170000000001,"15min":422.97000000000003},"min":{"1min":8.3000000000000007,"5min":6.8929999999999998,"15min":6.8929999999999998},"max":{"1min":25.968,"5min":23610.142,"15min":23610.142},"last":9.9930000000000003}]},{"osd":6,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":14.585000000000001,"5min":21.475000000000001,"15min":47.896000000000001},"min":{"1min":7.5540000000000003,"5min":7.5540000000000003,"15min":7.5540000000000003},"max":{"1min":25.594000000000001,"5min":263.983,"15min":450.17500000000001},"last":15.656000000000001},{"interface":"front","average":{"1min":17.135999999999999,"5min":23.904,"15min":53.079000000000001},"min":{"1min":12.586,"5min":8.1969999999999992,"15min":8.1969999999999992},"max":{"1min":29.440999999999999,"5min":202.196,"15min":417.303},"last":19.212}]},{"osd":7,"last update":"Sat Jul 29 14:06:05 2023","interfaces":[{"interface":"back","average":{"1min":17.875,"5min":24.571000000000002,"15min":44.404000000000003},"min":{"1min":12.962999999999999,"5min":11.222,"15min":10.602},"max":{"1min":28.838000000000001,"5min":284.48000000000002,"15min":341.30500000000001},"last":17.253},{"interface":"front","average":{"1min":16.516999999999999,"5min":24.103999999999999,"15min":50.014000000000003},"min":{"1min":9.5359999999999996,"5min":9.5359999999999996,"15min":9.5359999999999996},"max":{"1min":28.317,"5min":176.96299999999999,"15min":329.86500000000001},"last":17.986999999999998}]}]},{"osd":7,"up_from":650,"seq":2791728742811,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59436,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312404,"statfs":{"total":96636764160,"available":96575901696,"internally_reserved":0,"allocated":6393856,"data_stored":3446908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 29 14:03:45 2023","interfaces":[{"interface":"back","average":{"1min":11.94,"5min":13.635,"15min":23.015000000000001},"min":{"1min":5.5890000000000004,"5min":5.4539999999999997,"15min":5.4539999999999997},"max":{"1min":28.5,"5min":75.968000000000004,"15min":293.94499999999999},"last":12.878},{"interface":"front","average":{"1min":14.968999999999999,"5min":18.658999999999999,"15min":28.620000000000001},"min":{"1min":6.8179999999999996,"5min":6.8179999999999996,"15min":6.8179999999999996},"max":{"1min":29.143999999999998,"5min":116.952,"15min":358.267},"last":7.1840000000000002}]},{"osd":1,"last update":"Sat Jul 29 14:05:54 2023","interfaces":[{"interface":"back","average":{"1min":9.4100000000000001,"5min":1105.6959999999999,"15min":387.80799999999999},"min":{"1min":6.4740000000000002,"5min":5.1219999999999999,"15min":4.6660000000000004},"max":{"1min":13.821,"5min":27546.963,"15min":27546.963},"last":11.609999999999999},{"interface":"front","average":{"1min":12.48,"5min":1109.259,"15min":384.47300000000001},"min":{"1min":6.4139999999999997,"5min":6.4139999999999997,"15min":5.5010000000000003},"max":{"1min":16.233000000000001,"5min":27546.280999999999,"15min":27546.280999999999},"last":8.9019999999999992}]},{"osd":2,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":13.449999999999999,"5min":17.512,"15min":25.91},"min":{"1min":9.0869999999999997,"5min":9.0869999999999997,"15min":7.9100000000000001},"max":{"1min":15.819000000000001,"5min":103.09,"15min":461.83800000000002},"last":8.1690000000000005},{"interface":"front","average":{"1min":8.734,"5min":13.756,"15min":21.891999999999999},"min":{"1min":7.0039999999999996,"5min":6.6239999999999997,"15min":6.6239999999999997},"max":{"1min":13.706,"5min":181.97900000000001,"15min":385.40800000000002},"last":10.122999999999999}]},{"osd":3,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":10.454000000000001,"5min":17.827999999999999,"15min":1298.6089999999999},"min":{"1min":7.8300000000000001,"5min":7.6660000000000004,"15min":7.6660000000000004},"max":{"1min":16.02,"5min":242.197,"15min":34550.838000000003},"last":16.628},{"interface":"front","average":{"1min":15.295,"5min":23.765999999999998,"15min":1309.0609999999999},"min":{"1min":12.576000000000001,"5min":8.1509999999999998,"15min":6.9550000000000001},"max":{"1min":18.369,"5min":243.06399999999999,"15min":34556.381000000001},"last":11.01}]},{"osd":4,"last update":"Sat Jul 29 14:05:54 2023","interfaces":[{"interface":"back","average":{"1min":11.284000000000001,"5min":1071.366,"15min":900.10199999999998},"min":{"1min":6.1319999999999997,"5min":6.1319999999999997,"15min":6.1319999999999997},"max":{"1min":18.895,"5min":25146.41,"15min":27891.68},"last":18.477},{"interface":"front","average":{"1min":15.611000000000001,"5min":1099.066,"15min":916.32100000000003},"min":{"1min":7.5990000000000002,"5min":7.5990000000000002,"15min":7.2699999999999996},"max":{"1min":21.300999999999998,"5min":25434.151999999998,"15min":27883.419999999998},"last":14.747999999999999}]},{"osd":5,"last update":"Sat Jul 29 14:06:08 2023","interfaces":[{"interface":"back","average":{"1min":17.609000000000002,"5min":23.239999999999998,"15min":66.055999999999997},"min":{"1min":13.454000000000001,"5min":11.215,"15min":10.683},"max":{"1min":23.506,"5min":149.554,"15min":1265.123},"last":14.101000000000001},{"interface":"front","average":{"1min":12.923,"5min":22.481000000000002,"15min":69.558999999999997},"min":{"1min":6.7140000000000004,"5min":6.7140000000000004,"15min":6.7140000000000004},"max":{"1min":20.945,"5min":255.23699999999999,"15min":1234.9300000000001},"last":18.096}]},{"osd":6,"last update":"Sat Jul 29 14:05:48 2023","interfaces":[{"interface":"back","average":{"1min":15.662000000000001,"5min":20.847000000000001,"15min":24.681000000000001},"min":{"1min":11.005000000000001,"5min":9.9740000000000002,"15min":7.8689999999999998},"max":{"1min":49.978000000000002,"5min":95.558999999999997,"15min":243.66499999999999},"last":17.475000000000001},{"interface":"front","average":{"1min":21.350999999999999,"5min":25.521999999999998,"15min":30.802},"min":{"1min":16.145,"5min":12.111000000000001,"15min":10.542},"max":{"1min":79.692999999999998,"5min":148.334,"15min":454.68799999999999},"last":13.507999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-29T14:06:37.017 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-07-29T14:06:37.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.122+0000 7f303527c640 10 monclient: get_monmap_and_config 2023-07-29T14:06:37.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.122+0000 7f303527c640 10 monclient: build_initial_monmap 2023-07-29T14:06:37.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.122+0000 7f303527c640 10 monclient: monmap: 2023-07-29T14:06:37.123 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:06:37.123 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:37.124 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T14:06:37.123738+0000 2023-07-29T14:06:37.124 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T14:06:37.123738+0000 2023-07-29T14:06:37.124 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:06:37.124 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:37.124 INFO:teuthology.orchestra.run.smithi040.stderr:0: v1:172.21.15.40:6789/0 mon.noname-a-legacy 2023-07-29T14:06:37.124 INFO:teuthology.orchestra.run.smithi040.stderr:1: v1:172.21.15.45:6789/0 mon.noname-c-legacy 2023-07-29T14:06:37.124 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-b 2023-07-29T14:06:37.125 INFO:teuthology.orchestra.run.smithi040.stderr:3: v2:172.21.15.40:3300/0 mon.noname-a 2023-07-29T14:06:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:4: v2:172.21.15.45:3300/0 mon.noname-c 2023-07-29T14:06:37.126 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 1 Processor -- start 2023-07-29T14:06:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 1 -- start start 2023-07-29T14:06:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 10 monclient: init 2023-07-29T14:06:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 10 monclient: _reopen_session rank -1 2023-07-29T14:06:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2023-07-29T14:06:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f30300b74a0 0x7f30300aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 10 monclient(hunting): picked mon.noname-c con 0x7f30300b74a0 addr v2:172.21.15.45:3300/0 2023-07-29T14:06:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f303013dde0 0x7f303013e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 10 monclient(hunting): picked mon.noname-a con 0x7f303013dde0 addr v2:172.21.15.40:3300/0 2023-07-29T14:06:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30301445e0 0x7f3030144a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 10 monclient(hunting): picked mon.noname-b con 0x7f30301445e0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:06:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30300b7870 con 0x7f303013dde0 2023-07-29T14:06:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303013e760 con 0x7f30300b74a0 2023-07-29T14:06:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30300a91b0 con 0x7f30301445e0 2023-07-29T14:06:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 10 monclient(hunting): _renew_subs 2023-07-29T14:06:37.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.123+0000 7f303527c640 10 monclient(hunting): authenticate will time out at 3719.256887s 2023-07-29T14:06:37.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.124+0000 7f302f7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30301445e0 0x7f3030144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:37.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.124+0000 7f302effd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f30300b74a0 0x7f30300aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.124+0000 7f302f7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30301445e0 0x7f3030144a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35306/0 (socket says 172.21.15.40:35306) 2023-07-29T14:06:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.124+0000 7f302f7fe640 1 -- 172.21.15.40:0/1479948838 learned_addr learned my addr 172.21.15.40:0/1479948838 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:06:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.124+0000 7f302f7fe640 10 monclient(hunting): get_auth_request con 0x7f30301445e0 auth_method 0 2023-07-29T14:06:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.124+0000 7f302f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.124+0000 7f302f7fe640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.124+0000 7f302f7fe640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.125+0000 7f302effd640 10 monclient(hunting): get_auth_request con 0x7f30300b74a0 auth_method 0 2023-07-29T14:06:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.125+0000 7f302effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.125+0000 7f302effd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.125+0000 7f302effd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.125+0000 7f302f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.125+0000 7f302f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:37.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.125+0000 7f302f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:37.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.125+0000 7f302effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:37.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.125+0000 7f302effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:37.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.125+0000 7f302effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:37.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.126+0000 7f302e7fc640 1 --2- 172.21.15.40:0/1479948838 >> v2:172.21.15.40:3300/0 conn(0x7f303013dde0 0x7f303013e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:37.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.127+0000 7f302f7fe640 10 monclient(hunting): handle_auth_done global_id 16454 payload 274 2023-07-29T14:06:37.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.127+0000 7f302f7fe640 1 -- 172.21.15.40:0/1479948838 >> v2:172.21.15.45:3300/0 conn(0x7f30300b74a0 msgr2=0x7f30300aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:06:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.127+0000 7f302f7fe640 1 --2- 172.21.15.40:0/1479948838 >> v2:172.21.15.45:3300/0 conn(0x7f30300b74a0 0x7f30300aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.127+0000 7f302f7fe640 1 -- 172.21.15.40:0/1479948838 >> v2:172.21.15.40:3300/0 conn(0x7f303013dde0 msgr2=0x7f303013e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-29T14:06:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.127+0000 7f302f7fe640 1 --2- 172.21.15.40:0/1479948838 >> v2:172.21.15.40:3300/0 conn(0x7f303013dde0 0x7f303013e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.127+0000 7f302f7fe640 10 monclient: _finish_hunting 0 2023-07-29T14:06:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.127+0000 7f302f7fe640 1 monclient: found mon.noname-b 2023-07-29T14:06:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.127+0000 7f302f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:06:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.127+0000 7f302f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.40:3301/0 2023-07-29T14:06:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.127+0000 7f302f7fe640 1 -- 172.21.15.40:0/1479948838 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30300a9490 con 0x7f30301445e0 2023-07-29T14:06:37.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.129+0000 7f302f7fe640 1 --2- 172.21.15.40:0/1479948838 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30301445e0 0x7f3030144a60 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 crypto rx=0x7f303006bd60 tx=0x7f302802b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c3cbce45e4f4ca server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:06:37.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 1 -- 172.21.15.40:0/1479948838 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30280317e0 con 0x7f30301445e0 2023-07-29T14:06:37.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:37.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-07-29T14:06:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 10 monclient: dump: 2023-07-29T14:06:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:37.164 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 10 monclient: _finish_auth 0 2023-07-29T14:06:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 10 monclient: _check_auth_tickets 2023-07-29T14:06:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:06:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 1 -- 172.21.15.40:0/1479948838 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3028031940 con 0x7f30301445e0 2023-07-29T14:06:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:06:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f303527c640 5 monclient: authenticate success, global_id 16454 2023-07-29T14:06:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 1 -- 172.21.15.40:0/1479948838 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3028031cd0 con 0x7f30301445e0 2023-07-29T14:06:37.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:37.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:06:37.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f302dffb640 10 monclient: dump: 2023-07-29T14:06:37.165 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:37.166 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:37.166 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:37.166 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:37.166 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:37.166 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:37.166 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f303527c640 10 monclient: get_monmap_and_config success 2023-07-29T14:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.135+0000 7f303527c640 4 set_mon_vals no callback set 2023-07-29T14:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.136+0000 7f303527c640 10 monclient: shutdown 2023-07-29T14:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.136+0000 7f303527c640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:06:37.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.136+0000 7f303527c640 1 -- 172.21.15.40:0/1479948838 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30301445e0 msgr2=0x7f3030144a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:37.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.136+0000 7f303527c640 1 --2- 172.21.15.40:0/1479948838 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30301445e0 0x7f3030144a60 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 crypto rx=0x7f303006bd60 tx=0x7f302802b880 comp rx=0 tx=0).stop 2023-07-29T14:06:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.136+0000 7f303527c640 1 -- 172.21.15.40:0/1479948838 shutdown_connections 2023-07-29T14:06:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.136+0000 7f303527c640 1 --2- 172.21.15.40:0/1479948838 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30301445e0 0x7f3030144a60 unknown :-1 s=CLOSED pgs=3246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.136+0000 7f303527c640 1 --2- 172.21.15.40:0/1479948838 >> v2:172.21.15.40:3300/0 conn(0x7f303013dde0 0x7f303013e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.136+0000 7f303527c640 1 --2- 172.21.15.40:0/1479948838 >> v2:172.21.15.45:3300/0 conn(0x7f30300b74a0 0x7f30300aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.136+0000 7f303527c640 1 -- 172.21.15.40:0/1479948838 >> 172.21.15.40:0/1479948838 conn(0x7f3030065b00 msgr2=0x7f3030065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.136+0000 7f303527c640 1 -- 172.21.15.40:0/1479948838 shutdown_connections 2023-07-29T14:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.136+0000 7f303527c640 1 -- 172.21.15.40:0/1479948838 wait complete. 2023-07-29T14:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.137+0000 7f303527c640 10 monclient: build_initial_monmap 2023-07-29T14:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.137+0000 7f303527c640 10 monclient: monmap: 2023-07-29T14:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 0 2023-07-29T14:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-07-29T14:06:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 0.000000 2023-07-29T14:06:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:created 0.000000 2023-07-29T14:06:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 0 (unknown) 2023-07-29T14:06:37.173 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:37.174 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.noname-a 2023-07-29T14:06:37.174 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.noname-b 2023-07-29T14:06:37.175 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.noname-c 2023-07-29T14:06:37.175 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:37.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.137+0000 7f303527c640 1 librados: starting msgr at 2023-07-29T14:06:37.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.137+0000 7f303527c640 1 librados: starting objecter 2023-07-29T14:06:37.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.137+0000 7f303527c640 1 Processor -- start 2023-07-29T14:06:37.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 1 -- start start 2023-07-29T14:06:37.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 1 librados: setting wanted keys 2023-07-29T14:06:37.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 1 librados: calling monclient init 2023-07-29T14:06:37.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 10 monclient: init 2023-07-29T14:06:37.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 10 monclient: _reopen_session rank -1 2023-07-29T14:06:37.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 10 monclient: _add_conns ranks=[2,1,0] 2023-07-29T14:06:37.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30300b74a0 0x7f30300aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:37.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 10 monclient(hunting): picked mon.noname-c con 0x7f30300b74a0 addr [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] 2023-07-29T14:06:37.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f303013dde0 0x7f30301d97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:37.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 10 monclient(hunting): picked mon.noname-b con 0x7f303013dde0 addr [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2023-07-29T14:06:37.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30301445e0 0x7f30301d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:37.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 10 monclient(hunting): picked mon.noname-a con 0x7f30301445e0 addr [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2023-07-29T14:06:37.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:37.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30301d9280 con 0x7f30301445e0 2023-07-29T14:06:37.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:37.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30300afbc0 con 0x7f303013dde0 2023-07-29T14:06:37.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 10 monclient(hunting): start opening mon connection 2023-07-29T14:06:37.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30300afd80 con 0x7f30300b74a0 2023-07-29T14:06:37.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 10 monclient(hunting): _renew_subs 2023-07-29T14:06:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.138+0000 7f303527c640 10 monclient(hunting): authenticate will time out at 3719.271693s 2023-07-29T14:06:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302effd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30300b74a0 0x7f30300aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f303013dde0 0x7f30301d97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302effd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30300b74a0 0x7f30300aabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35312/0 (socket says 172.21.15.40:35312) 2023-07-29T14:06:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302effd640 1 -- 172.21.15.40:0/1650786015 learned_addr learned my addr 172.21.15.40:0/1650786015 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-29T14:06:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302effd640 10 monclient(hunting): get_auth_request con 0x7f30300b74a0 auth_method 0 2023-07-29T14:06:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302effd640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302effd640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302e7fc640 10 monclient(hunting): get_auth_request con 0x7f303013dde0 auth_method 0 2023-07-29T14:06:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-07-29T14:06:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302e7fc640 10 monclient(hunting): _init_auth method 2 2023-07-29T14:06:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.139+0000 7f302e7fc640 10 monclient(hunting): _init_auth creating new auth 2023-07-29T14:06:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.140+0000 7f302effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.140+0000 7f302effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.140+0000 7f302effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.140+0000 7f302e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-07-29T14:06:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.140+0000 7f302e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-07-29T14:06:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.140+0000 7f302e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-07-29T14:06:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.142+0000 7f302effd640 10 monclient(hunting): handle_auth_done global_id 16457 payload 824 2023-07-29T14:06:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.142+0000 7f302effd640 1 -- 172.21.15.40:0/1650786015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f303013dde0 msgr2=0x7f30301d97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.142+0000 7f302effd640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f303013dde0 0x7f30301d97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.142+0000 7f302effd640 1 -- 172.21.15.40:0/1650786015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30301445e0 msgr2=0x7f30301d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:37.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.142+0000 7f302effd640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30301445e0 0x7f30301d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.142+0000 7f302effd640 10 monclient: _finish_hunting 0 2023-07-29T14:06:37.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.142+0000 7f302effd640 1 monclient: found mon.noname-c 2023-07-29T14:06:37.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.142+0000 7f302effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-07-29T14:06:37.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.142+0000 7f302effd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.40:3301/0 2023-07-29T14:06:37.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.142+0000 7f302effd640 1 -- 172.21.15.40:0/1650786015 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30280094d0 con 0x7f30300b74a0 2023-07-29T14:06:37.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.145+0000 7f302effd640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30300b74a0 0x7f30300aabf0 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 crypto rx=0x7f302000efc0 tx=0x7f302000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:06:37.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.145+0000 7f30177fe640 10 client.?.objecter ms_handle_connect 0x7f30300b74a0 2023-07-29T14:06:37.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.145+0000 7f30177fe640 10 client.?.objecter resend_mon_ops 2023-07-29T14:06:37.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 1 -- 172.21.15.40:0/1650786015 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020014480 con 0x7f30300b74a0 2023-07-29T14:06:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-07-29T14:06:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 10 monclient: dump: 2023-07-29T14:06:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:37.194 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 10 monclient: _finish_auth 0 2023-07-29T14:06:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 10 monclient: _check_auth_tickets 2023-07-29T14:06:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-07-29T14:06:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 1 -- 172.21.15.40:0/1650786015 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302000f040 con 0x7f30300b74a0 2023-07-29T14:06:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f303527c640 5 monclient: authenticate success, global_id 16457 2023-07-29T14:06:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f303527c640 10 monclient: _renew_subs 2023-07-29T14:06:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f303527c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:06:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f303527c640 1 -- 172.21.15.40:0/1650786015 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30300b0fd0 con 0x7f30300b74a0 2023-07-29T14:06:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 10 monclient: handle_config config(0 keys) v1 2023-07-29T14:06:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 1 -- 172.21.15.40:0/1650786015 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020014800 con 0x7f30300b74a0 2023-07-29T14:06:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-07-29T14:06:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-07-29T14:06:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f30177fe640 10 monclient: dump: 2023-07-29T14:06:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:epoch 1 2023-07-29T14:06:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:fsid d11937c2-52a1-49ac-aadc-1ae254eda669 2023-07-29T14:06:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:last_changed 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:created 2023-07-29T13:26:33.423641+0000 2023-07-29T14:06:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:min_mon_release 18 (reef) 2023-07-29T14:06:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:election_strategy: 1 2023-07-29T14:06:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-07-29T14:06:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2023-07-29T14:06:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2023-07-29T14:06:37.199 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-29T14:06:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f303527c640 10 client.16457.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-07-29T14:06:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f303527c640 10 monclient: _renew_subs 2023-07-29T14:06:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f302d7fa640 4 set_mon_vals no callback set 2023-07-29T14:06:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f303527c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:06:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f303527c640 1 -- 172.21.15.40:0/1650786015 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30300ab310 con 0x7f30300b74a0 2023-07-29T14:06:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.148+0000 7f303527c640 1 librados: init done 2023-07-29T14:06:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.149+0000 7f303527c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-07-29T14:06:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.149+0000 7f303527c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:06:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.149+0000 7f303527c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:06:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.149+0000 7f303527c640 1 -- 172.21.15.40:0/1650786015 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303004e000 con 0x7f30300b74a0 2023-07-29T14:06:37.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.412+0000 7f30177fe640 1 -- 172.21.15.40:0/1650786015 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 7) v1 ==== 88209+0+0 (secure 0 0 0) 0x7f3020014960 con 0x7f30300b74a0 2023-07-29T14:06:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.412+0000 7f30177fe640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f301c06bf60 0x7f301c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-29T14:06:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.412+0000 7f302e7fc640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f301c06bf60 0x7f301c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-29T14:06:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.412+0000 7f302e7fc640 10 monclient: get_auth_request con 0x7f301c06bf60 auth_method 0 2023-07-29T14:06:37.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.413+0000 7f30177fe640 10 client.16457.objecter ms_handle_connect 0x7f301c06bf60 2023-07-29T14:06:37.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.413+0000 7f302e7fc640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f301c06bf60 0x7f301c06e410 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f30300b46a0 tx=0x7f3024008040 comp rx=0 tx=0).ready entity=mgr.4152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-29T14:06:37.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.429+0000 7f30177fe640 1 -- 172.21.15.40:0/1650786015 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1004..1004 src has 571..1004) v4 ==== 6193+0+0 (secure 0 0 0) 0x7f302005cbe0 con 0x7f30300b74a0 2023-07-29T14:06:37.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.429+0000 7f30177fe640 10 client.16457.objecter ms_dispatch 0x7f3030000b90 osd_map(1004..1004 src has 571..1004) v4 2023-07-29T14:06:37.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.429+0000 7f30177fe640 3 client.16457.objecter handle_osd_map got epochs [1004,1004] > 0 2023-07-29T14:06:37.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.429+0000 7f30177fe640 3 client.16457.objecter handle_osd_map decoding full epoch 1004 2023-07-29T14:06:37.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.429+0000 7f30177fe640 20 client.16457.objecter dump_active .. 0 homeless 2023-07-29T14:06:37.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.429+0000 7f30177fe640 1 -- 172.21.15.40:0/1650786015 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176809 (secure 0 0 0) 0x7f30200589f0 con 0x7f30300b74a0 2023-07-29T14:06:37.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.429+0000 7f30177fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-07-29T14:06:37.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.429+0000 7f30177fe640 10 monclient: _finish_command 1 = system:0 2023-07-29T14:06:37.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.634+0000 7f303527c640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2023-07-29T14:06:37.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.634+0000 7f303527c640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2023-07-29T14:06:37.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.634+0000 7f303527c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.40:3301/0 2023-07-29T14:06:37.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.634+0000 7f303527c640 1 -- 172.21.15.40:0/1650786015 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f30301d9440 con 0x7f30300b74a0 2023-07-29T14:06:37.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.868+0000 7f30177fe640 1 -- 172.21.15.40:0/1650786015 <== mon.2 v2:172.21.15.40:3301/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f30200589f0 con 0x7f30300b74a0 2023-07-29T14:06:37.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.868+0000 7f30177fe640 10 monclient: handle_config config(1 keys) v1 2023-07-29T14:06:37.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.868+0000 7f302d7fa640 4 set_mon_vals no callback set 2023-07-29T14:06:37.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.868+0000 7f302d7fa640 10 set_mon_vals mon_health_to_clog = false 2023-07-29T14:06:37.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.868+0000 7f30177fe640 1 -- 172.21.15.40:0/1650786015 <== mon.2 v2:172.21.15.40:3301/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3020058580 con 0x7f30300b74a0 2023-07-29T14:06:37.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.868+0000 7f30177fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2023-07-29T14:06:37.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.868+0000 7f30177fe640 10 monclient: _finish_command 2 = system:0 2023-07-29T14:06:37.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 10 librados: watch_flush enter 2023-07-29T14:06:37.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 10 librados: watch_flush exit 2023-07-29T14:06:37.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 20 client.16457.objecter shutdown clearing up homeless session... 2023-07-29T14:06:37.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 10 client.16457.objecter successfully canceled tick 2023-07-29T14:06:37.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 -- 172.21.15.40:0/1650786015 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f301c06bf60 msgr2=0x7f301c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:37.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f301c06bf60 0x7f301c06e410 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f30300b46a0 tx=0x7f3024008040 comp rx=0 tx=0).stop 2023-07-29T14:06:37.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 10 monclient: shutdown 2023-07-29T14:06:37.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 20 monclient: shutdown discarding 0 pending message(s) 2023-07-29T14:06:37.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 -- 172.21.15.40:0/1650786015 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30300b74a0 msgr2=0x7f30300aabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-29T14:06:37.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30300b74a0 0x7f30300aabf0 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 crypto rx=0x7f302000efc0 tx=0x7f302000c490 comp rx=0 tx=0).stop 2023-07-29T14:06:37.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 -- 172.21.15.40:0/1650786015 shutdown_connections 2023-07-29T14:06:37.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.45:6800/84611,v1:172.21.15.45:6801/84611] conn(0x7f301c06bf60 0x7f301c06e410 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f303013dde0 0x7f30301d97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30301445e0 0x7f30301d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 --2- 172.21.15.40:0/1650786015 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f30300b74a0 0x7f30300aabf0 unknown :-1 s=CLOSED pgs=3247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-29T14:06:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 -- 172.21.15.40:0/1650786015 >> 172.21.15.40:0/1650786015 conn(0x7f3030065b00 msgr2=0x7f303006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-29T14:06:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 -- 172.21.15.40:0/1650786015 shutdown_connections 2023-07-29T14:06:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 -- 172.21.15.40:0/1650786015 wait complete. 2023-07-29T14:06:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-29T14:06:37.871+0000 7f303527c640 1 librados: shutdown 2023-07-29T14:06:37.887 INFO:teuthology.misc:Shutting down mds daemons... 2023-07-29T14:06:37.887 INFO:teuthology.misc:Shutting down osd daemons... 2023-07-29T14:06:37.888 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-07-29T14:06:37.888 INFO:teuthology.orchestra.run:waiting for 300 2023-07-29T14:06:37.902 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T14:06:37.892+0000 b47d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 85550) UID: 0 2023-07-29T14:06:37.902 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-07-29T14:06:37.893+0000 b47d640 -1 osd.0 1004 *** Got signal Terminated *** 2023-07-29T14:06:43.994 INFO:tasks.ceph.osd.0:Stopped 2023-07-29T14:06:43.995 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-07-29T14:06:43.996 INFO:teuthology.orchestra.run:waiting for 300 2023-07-29T14:06:44.022 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T14:06:43.999+0000 b47d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 85555) UID: 0 2023-07-29T14:06:44.023 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-07-29T14:06:44.000+0000 b47d640 -1 osd.1 1006 *** Got signal Terminated *** 2023-07-29T14:06:44.658 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-07-29T14:06:47.303 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:42.636 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:47.304 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:42.639 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:47.323 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:42.657 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:47.352 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:42.638 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:47.353 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:42.688 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:47.354 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:42.639 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:47.354 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:42.689 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:47.374 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:42.658 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:47.375 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:42.708 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:50.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-07-29T14:06:52.234 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:47.570 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:52.237 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:47.573 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:52.247 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:47.583 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:52.287 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:47.572 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:52.290 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:47.575 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:52.291 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:47.622 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:52.291 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:47.625 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:52.301 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:47.636 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:52.301 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:47.586 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:57.316 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:52.651 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:57.320 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:52.654 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:57.326 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:52.611 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:57.332 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:52.614 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:57.337 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:39:52.672 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:57.346 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:39:52.630 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:57.374 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:52.709 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:57.378 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:52.713 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:06:57.392 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:39:52.727 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:07.333 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:02.668 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:07.334 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:02.670 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:07.345 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:02.680 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:07.380 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:02.666 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:07.381 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:02.716 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:07.386 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:02.719 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:07.386 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:02.669 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:07.395 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:02.681 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:07.403 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:02.733 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:12.297 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:07.632 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:12.347 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:07.682 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:12.352 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:07.685 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:12.353 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:07.687 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:12.353 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:07.635 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:12.353 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:07.638 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:12.363 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:07.698 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:12.368 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:07.651 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:12.376 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:07.708 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:27.410 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:22.744 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:27.410 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:22.746 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:27.423 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:22.758 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:27.459 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:22.794 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:27.464 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:22.799 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:27.464 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:22.750 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:27.467 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:22.753 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:27.478 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:22.813 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:27.482 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:22.767 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:32.413 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:27.748 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:32.415 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:27.750 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:32.429 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:27.764 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:32.466 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:27.801 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:32.467 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:27.752 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:32.467 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:27.803 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:32.469 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:27.755 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:32.483 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:27.766 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:32.484 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:27.815 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:47.457 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:42.792 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:47.460 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:42.796 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:47.486 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:42.821 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:47.512 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:42.797 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:47.514 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:42.800 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:47.520 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:42.856 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:47.522 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:42.858 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:47.532 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:42.817 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:47.542 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:42.875 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:52.467 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:47.801 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:52.470 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:47.806 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:52.483 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:47.818 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:52.522 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:47.807 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:52.523 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:47.859 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:52.525 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:47.811 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:52.527 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:47.863 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:52.537 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:47.822 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:52.539 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:47.874 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:58.588 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:53.924 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:58.601 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:53.886 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:58.601 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:53.936 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:58.634 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:53.969 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:58.645 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:53.979 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:58.645 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:53.931 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:58.657 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:40:53.987 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:58.662 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:40:53.948 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:07:58.664 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:40:53.999 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:07.509 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:02.844 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:07.513 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:02.847 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:07.527 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:02.862 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:07.561 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:02.847 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:07.563 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:02.849 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:07.580 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:02.861 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:07.588 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:02.923 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:07.590 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:02.926 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:07.606 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:02.941 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:12.518 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:07.853 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:12.522 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:07.857 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:12.535 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:07.870 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:12.572 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:07.907 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:12.575 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:07.861 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:12.575 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:07.911 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:12.577 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:07.863 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:12.585 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:07.921 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:12.588 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:07.873 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:27.575 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:22.908 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:27.577 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:22.913 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:27.598 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:22.929 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:27.625 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:22.911 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:27.629 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:22.914 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:27.644 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:22.929 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:27.662 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:22.995 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:27.663 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:22.996 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:27.677 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:23.013 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:32.588 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:27.923 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:32.591 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:27.926 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:32.603 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:27.938 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:32.640 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:27.975 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:32.641 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:27.926 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:32.642 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:27.977 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:32.644 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:27.929 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:32.652 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:27.987 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:32.654 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:27.940 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:47.629 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:42.964 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:47.630 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:42.966 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:47.644 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:42.979 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:47.687 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:43.023 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:47.688 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:42.973 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:47.689 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:43.025 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:47.689 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:42.975 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:47.703 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:42.988 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:47.704 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:43.039 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:52.645 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:47.980 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:52.647 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:47.982 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:52.657 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:41:47.993 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:52.695 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:47.980 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:52.698 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:47.982 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:52.708 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:41:47.994 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:52.717 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:48.052 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:52.722 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:48.057 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:08:52.733 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:41:48.068 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:07.681 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:03.016 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:07.684 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:03.019 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:07.725 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:03.060 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:07.736 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:03.021 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:07.739 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:03.074 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:07.739 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:03.023 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:07.747 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:03.082 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:07.753 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:03.038 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:07.762 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:03.097 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:12.691 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:08.026 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:12.695 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:08.030 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:12.710 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:08.045 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:12.746 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:08.032 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:12.750 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:08.085 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:12.750 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:08.036 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:12.754 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:08.089 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:12.762 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:08.047 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:12.765 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:08.100 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:27.730 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:23.066 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:27.738 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:23.073 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:27.776 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:23.111 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:27.790 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:23.075 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:27.791 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:23.077 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:27.792 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:23.127 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:27.798 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:23.133 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:27.806 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:23.091 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:27.813 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:23.149 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:32.751 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:28.087 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:32.753 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:28.089 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:32.767 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:28.102 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:32.800 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:28.136 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:32.805 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:28.090 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:32.805 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:28.140 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:32.806 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:28.092 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:32.816 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:28.152 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:32.817 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:28.102 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:47.792 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:43.128 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:47.795 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:43.130 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:47.808 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:43.143 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:47.846 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:43.132 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:47.847 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:43.182 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:47.849 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:43.134 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:47.849 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:43.185 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:47.864 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:43.149 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:47.865 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:43.200 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:52.802 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:48.137 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:52.806 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:48.142 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:52.826 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:42:48.160 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:52.861 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:48.196 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:52.862 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:48.197 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:52.864 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:48.150 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:52.868 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:48.153 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:52.873 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:42:48.209 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:09:52.878 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:42:48.164 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:07.846 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:03.181 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:07.850 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:03.186 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:07.866 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:03.202 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:07.903 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:03.189 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:07.905 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:03.240 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:07.905 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:03.190 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:07.907 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:03.243 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:07.919 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:03.204 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:07.924 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:03.258 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:12.813 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:08.149 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:12.865 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:08.200 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:12.880 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:08.215 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:12.915 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:08.251 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:12.918 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:08.254 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:12.919 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:08.204 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:12.921 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:08.207 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:12.929 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:08.265 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:12.935 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:08.218 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:27.993 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:23.329 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:27.995 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:23.331 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:28.010 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:23.345 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:28.049 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:23.334 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:28.049 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:23.385 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:28.050 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:23.336 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:28.050 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:23.386 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:28.064 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:23.350 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:28.065 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:23.401 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:32.897 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:28.232 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:32.946 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:28.281 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:32.947 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:28.282 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:32.949 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:28.285 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:32.964 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:28.299 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:32.966 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:28.302 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:33.003 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:28.289 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:33.004 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:28.290 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:33.014 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:28.299 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:48.028 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:43.363 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:48.055 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:43.390 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:48.075 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:43.410 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:48.084 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:43.420 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:48.085 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:43.370 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:48.085 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:43.421 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:48.086 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:43.371 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:48.099 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:43.385 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:48.104 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:43.440 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:53.020 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:48.356 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:53.025 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:48.360 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:53.039 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:43:48.374 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:53.075 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:48.360 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:53.075 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:48.411 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:53.077 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:48.363 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:53.079 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:48.414 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:53.090 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:43:48.376 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:10:53.094 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:43:48.429 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:05.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-07-29T14:11:06.390 INFO:tasks.ceph.osd.1:Stopped 2023-07-29T14:11:06.391 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-07-29T14:11:06.392 INFO:teuthology.orchestra.run:waiting for 300 2023-07-29T14:11:06.404 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T14:11:06.395+0000 b47d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 85560) UID: 0 2023-07-29T14:11:06.405 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-07-29T14:11:06.396+0000 b47d640 -1 osd.2 1010 *** Got signal Terminated *** 2023-07-29T14:11:08.215 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:03.551 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:08.218 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:03.553 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:08.234 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:03.569 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:08.267 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:03.602 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:08.268 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:03.604 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:08.277 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:03.562 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:08.278 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:03.563 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:08.285 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:03.621 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:08.293 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:03.579 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:10.573 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:05.908 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:10.575 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:05.910 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:10.588 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:05.924 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:10.625 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:05.911 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:10.628 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:05.914 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:10.629 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:05.964 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:10.630 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:05.966 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:10.643 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:05.928 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:10.650 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:05.985 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:11.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-07-29T14:11:11.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-07-29T14:11:12.493 INFO:tasks.ceph.osd.2:Stopped 2023-07-29T14:11:12.494 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-07-29T14:11:12.494 INFO:teuthology.orchestra.run:waiting for 300 2023-07-29T14:11:12.508 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T14:11:12.498+0000 b47d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 85562) UID: 0 2023-07-29T14:11:12.508 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-07-29T14:11:12.499+0000 b47d640 -1 osd.3 1013 *** Got signal Terminated *** 2023-07-29T14:11:13.207 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:08.542 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:13.212 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:08.547 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:13.222 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:08.558 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:13.263 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:08.598 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:13.265 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:08.551 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:13.267 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:08.553 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:13.269 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:08.604 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:13.277 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:08.562 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:13.279 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:08.615 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:17.391 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-07-29T14:11:17.391 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-07-29T14:11:18.596 INFO:tasks.ceph.osd.3:Stopped 2023-07-29T14:11:18.597 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-07-29T14:11:18.597 INFO:teuthology.orchestra.run:waiting for 300 2023-07-29T14:11:18.611 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T14:11:18.601+0000 b47d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 85173) UID: 0 2023-07-29T14:11:18.611 INFO:tasks.ceph.osd.4.smithi045.stderr:2023-07-29T14:11:18.602+0000 b47d640 -1 osd.4 1016 *** Got signal Terminated *** 2023-07-29T14:11:23.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-07-29T14:11:23.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-07-29T14:11:23.237 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:18.573 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:23.242 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:18.578 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:23.259 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:18.595 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:23.290 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:18.575 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:23.291 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:18.577 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:23.292 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:18.628 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:23.293 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:18.629 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:23.310 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:18.596 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:23.313 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:18.648 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:24.698 INFO:tasks.ceph.osd.4:Stopped 2023-07-29T14:11:24.699 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-07-29T14:11:24.699 INFO:teuthology.orchestra.run:waiting for 300 2023-07-29T14:11:24.716 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T14:11:24.703+0000 b47d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 85174) UID: 0 2023-07-29T14:11:24.716 INFO:tasks.ceph.osd.5.smithi045.stderr:2023-07-29T14:11:24.704+0000 b47d640 -1 osd.5 1018 *** Got signal Terminated *** 2023-07-29T14:11:28.903 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-07-29T14:11:28.904 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2023-07-29T14:11:30.801 INFO:tasks.ceph.osd.5:Stopped 2023-07-29T14:11:30.802 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-07-29T14:11:30.802 INFO:teuthology.orchestra.run:waiting for 300 2023-07-29T14:11:30.816 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T14:11:30.806+0000 b47d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 85179) UID: 0 2023-07-29T14:11:30.817 INFO:tasks.ceph.osd.6.smithi045.stderr:2023-07-29T14:11:30.807+0000 b47d640 -1 osd.6 1022 *** Got signal Terminated *** 2023-07-29T14:11:33.207 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:28.542 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:33.211 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:28.547 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:33.229 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:28.564 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:33.258 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:28.544 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:33.261 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:28.596 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:33.262 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:28.548 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:33.264 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:28.600 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:33.278 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:28.564 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:33.280 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:28.616 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:34.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-07-29T14:11:43.242 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:38.577 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:43.244 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:38.579 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:43.255 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:38.591 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:43.285 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:38.570 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:43.288 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:38.574 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:43.292 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:38.628 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:43.295 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:38.631 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:43.299 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:38.585 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:43.309 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:38.644 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:53.264 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:48.600 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:53.270 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:48.605 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:53.285 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:48.620 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:53.317 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:48.603 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:53.319 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:48.605 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:53.320 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:48.655 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:53.322 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:48.658 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:53.332 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:48.618 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:11:53.335 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:48.671 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:03.300 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:58.635 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:03.302 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:58.638 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:03.322 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:44:58.657 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:03.351 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:58.636 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:03.352 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:58.638 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:03.359 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:58.694 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:03.360 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:58.696 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:03.364 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:44:58.650 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:03.371 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:44:58.707 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:13.326 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:08.661 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:13.334 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:08.669 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:13.346 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:08.682 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:13.376 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:08.662 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:13.381 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:08.666 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:13.393 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:08.678 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:13.408 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:08.744 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:13.409 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:08.745 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:13.422 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:08.757 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:23.364 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:18.699 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:23.365 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:18.701 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:23.379 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:18.714 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:23.408 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:18.694 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:23.411 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:18.697 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:23.423 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:18.708 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:23.447 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:18.782 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:23.448 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:18.784 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:23.459 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:18.795 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:33.394 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:28.730 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:33.396 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:28.732 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:33.410 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:28.746 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:33.441 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:28.727 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:33.444 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:28.729 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:33.446 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:28.781 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:33.449 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:28.785 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:33.457 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:28.742 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:33.462 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:28.797 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:43.421 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:38.757 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:43.424 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:38.759 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:43.437 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:38.773 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:43.468 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:38.754 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:43.471 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:38.757 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:43.475 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:38.810 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:43.476 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:38.812 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:43.484 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:38.769 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:43.488 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:38.824 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:51.770 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:12:51.769+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:12:52.732 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:12:52.732+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:12:53.473 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:48.809 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:53.478 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:48.813 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:53.496 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:48.831 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:53.526 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:48.812 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:53.529 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:48.814 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:53.529 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:48.865 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:53.531 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:48.867 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:53.545 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:48.831 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:53.548 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:48.884 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:12:53.775 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:12:53.775+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:12:54.769 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:12:54.768+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:12:55.783 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:12:55.782+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:12:56.783 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:12:56.782+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:12:57.819 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:12:57.819+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:12:58.837 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:12:58.836+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:12:59.881 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:12:59.881+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:00.907 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:00.906+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:01.941 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:01.941+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:02.971 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:02.971+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:03.512 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:58.847 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:03.515 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:58.850 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:03.530 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:45:58.866 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:03.563 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:58.849 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:03.564 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:58.850 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:03.565 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:58.901 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:03.568 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:58.904 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:03.577 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:45:58.863 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:03.581 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:45:58.916 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:03.923 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:03.922+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:04.944 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:04.943+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:05.931 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:05.930+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:06.976 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:06.975+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:08.025 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:08.025+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:09.020 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:09.020+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:10.006 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:10.004+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:11.043 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:11.042+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:12.069 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:12.068+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:13.107 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:13.105+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:13.540 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:08.875 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:13.544 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:08.880 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:13.560 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:08.896 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:13.592 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:08.878 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:13.595 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:08.881 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:13.596 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:08.932 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:13.598 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:08.934 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:13.607 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:08.893 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:13.611 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:08.947 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:14.157 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:14.155+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:15.176 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:15.175+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:16.131 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:16.130+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:17.114 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:17.112+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:18.132 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:18.130+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:19.163 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:19.161+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:20.201 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:20.198+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:21.181 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:21.180+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:22.201 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:22.199+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:23.194 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:23.192+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:23.571 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:18.906 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:23.575 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:18.911 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:23.592 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:18.927 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:23.621 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:18.906 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:23.625 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:18.911 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:23.626 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:18.962 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:23.629 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:18.965 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:23.638 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:18.924 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:23.642 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:18.978 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:24.208 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:24.206+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:25.185 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:25.184+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:26.221 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:26.220+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:27.209 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:27.208+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:28.230 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:28.228+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:29.219 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:29.218+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:30.172 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:30.170+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:31.212 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:31.210+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:32.194 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:32.192+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:33.227 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:33.226+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:33.606 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:28.942 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:33.608 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:28.944 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:33.621 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:28.956 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:33.651 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:28.937 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:33.654 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:28.940 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:33.656 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:28.992 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:33.659 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:28.995 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:33.666 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:28.952 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:33.671 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:29.007 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:34.246 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:34.245+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:35.207 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:35.207+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:36.186 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:36.185+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:37.175 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:37.175+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:38.156 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:38.155+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:39.181 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:39.180+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:40.139 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:40.139+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:41.119 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:41.119+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:42.102 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:42.101+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:43.068 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:43.067+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:43.636 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:38.972 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:43.641 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:38.977 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:43.658 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:38.994 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:43.688 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:38.974 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:43.691 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:38.977 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:43.694 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:39.030 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:43.695 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:39.031 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:43.704 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:38.990 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:43.709 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:39.045 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:44.036 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:44.035+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:45.056 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:45.056+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:46.028 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:46.028+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:47.077 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:47.077+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:48.113 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:48.113+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:49.161 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:49.160+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:50.138 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:50.137+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:51.172 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:51.172+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:52.152 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:52.151+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:53.170 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:53.169+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:53.670 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:49.006 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:53.672 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:49.009 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:53.692 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:49.028 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:53.715 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:49.001 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:53.717 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:49.003 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:53.730 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:49.016 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:53.751 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:49.087 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:53.753 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:49.089 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:53.765 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:49.101 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:13:54.154 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:54.153+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:55.137 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:55.137+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:56.139 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:56.139+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:57.166 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:57.165+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:58.167 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:58.167+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:13:59.206 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:13:59.205+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:00.236 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:00.236+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:01.265 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:01.265+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:02.270 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:02.270+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:03.298 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:03.298+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:03.703 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:59.039 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:03.705 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:59.041 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:03.718 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:46:59.055 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:03.747 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:59.033 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:03.750 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:59.037 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:03.753 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:59.089 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:03.754 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:59.091 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:03.763 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:46:59.049 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:03.768 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:46:59.104 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:04.292 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:04.292+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:05.321 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:05.321+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:06.355 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:06.356+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:07.307 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:07.307+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:08.323 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:08.322+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:09.341 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:09.341+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:10.346 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:10.346+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:11.377 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:11.377+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:12.099 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:07.435 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:12.102 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:07.438 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:12.118 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:07.454 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:12.154 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:07.441 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:12.156 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:07.443 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:12.157 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:07.494 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:12.161 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:07.497 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:12.175 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:07.462 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:12.176 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:07.513 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:12.399 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:12.400+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:13.447 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:13.446+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:13.728 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:09.065 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:13.729 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:09.066 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:13.738 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:09.075 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:13.780 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:09.067 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:13.781 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:09.118 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:13.782 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:09.069 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:13.783 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:09.119 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:13.790 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:09.077 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:13.792 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:09.128 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:14.412 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:14.411+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:15.444 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:15.444+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:16.464 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:16.463+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:17.507 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:17.506+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:18.508 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:18.508+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:19.534 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:19.534+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:20.527 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:20.527+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:21.553 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:21.552+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:22.582 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:22.582+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:23.577 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:23.577+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:23.763 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:19.100 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:23.765 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:19.102 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:23.805 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:19.141 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:23.812 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:19.099 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:23.814 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:19.101 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:23.816 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:19.153 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:23.819 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:19.156 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:23.827 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:19.114 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:23.832 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:19.169 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:24.598 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:24.598+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:25.632 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:25.631+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:26.649 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:26.648+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:27.641 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:27.640+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:28.608 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:28.608+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:29.609 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:29.609+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:30.656 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:30.656+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:31.684 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:31.684+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:32.645 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:32.645+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:33.652 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:33.652+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:33.741 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:29.078 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:33.789 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:29.076 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:33.790 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:29.078 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:33.794 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:29.131 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:33.796 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:29.134 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:33.799 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:29.136 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:33.805 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:29.092 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:33.809 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:29.146 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:33.839 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:29.176 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:34.628 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:34.629+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:35.617 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:35.618+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:36.643 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:36.644+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:37.679 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:37.680+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:38.667 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:38.668+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:39.710 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:39.711+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:40.756 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:40.757+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:41.758 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:41.759+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:42.711 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:42.712+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:43.718 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:43.718+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:43.828 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:39.165 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:43.853 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:39.190 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:43.866 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:39.203 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:43.876 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:39.163 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:43.877 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:39.165 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:43.887 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:39.224 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:43.888 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:39.226 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:43.889 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:39.177 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:43.900 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:39.237 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:44.707 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:44.707+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:45.668 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:45.669+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:46.682 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:46.683+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:47.724 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:47.725+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:48.747 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:48.748+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:49.768 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:49.768+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:50.819 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:50.820+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:51.838 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:51.839+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:52.821 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:52.822+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:53.839 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:53.840+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:53.873 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:49.211 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:53.879 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:49.216 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:53.894 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:49.232 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:53.924 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:49.211 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:53.926 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:49.214 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:53.927 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:49.265 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:53.930 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:49.268 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:53.939 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:49.226 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:53.947 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:49.285 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:14:54.898 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:54.898+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:55.880 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:55.881+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:56.867 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:56.869+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:57.860 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:57.861+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:58.888 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:58.889+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:14:59.855 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:14:59.856+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:00.840 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:00.842+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:01.875 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:01.877+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:02.868 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:02.870+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:03.880 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:59.218 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:03.886 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:59.224 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:03.905 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:47:59.243 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:03.906 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:03.907+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:03.934 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:59.222 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:03.938 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:59.226 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:03.941 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:59.279 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:03.944 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:59.282 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:03.950 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:47:59.238 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:03.960 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:47:59.298 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:04.882 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:04.883+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:05.871 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:05.873+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:06.877 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:06.879+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:07.838 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:07.840+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:08.825 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:08.827+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:09.847 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:09.849+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:10.799 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:10.800+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:11.766 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:11.768+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:12.784 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:12.784+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:13.830 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:13.830+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:13.913 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:09.251 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:13.917 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:09.255 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:13.932 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:09.270 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:13.964 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:09.252 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:13.966 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:09.304 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:13.969 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:09.307 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:13.969 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:09.256 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:13.981 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:09.269 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:13.982 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:09.320 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:14.787 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:14.788+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:15.799 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:15.800+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:16.753 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:16.754+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:17.802 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:17.803+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:18.763 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:18.765+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:19.812 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:19.813+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:20.784 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:20.785+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:21.779 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:21.780+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:22.767 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:22.768+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:23.780 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:23.781+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:23.944 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:19.282 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:23.949 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:19.287 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:23.968 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:19.306 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:24.000 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:19.288 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:24.001 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:19.290 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:24.002 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:19.341 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:24.004 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:19.342 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:24.014 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:19.302 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:24.018 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:19.356 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:24.785 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:24.786+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:25.739 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:25.741+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:26.755 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:15:26.756+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:12:51.526351+0000) 2023-07-29T14:15:33.997 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:29.336 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:34.000 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:29.338 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:34.013 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:29.352 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:34.048 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:29.336 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:34.050 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:29.338 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:34.050 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:29.389 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:34.052 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:29.390 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:34.062 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:29.351 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:34.064 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:29.402 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:44.041 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:39.380 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:44.045 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:39.384 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:44.060 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:39.399 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:44.091 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:39.380 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:44.093 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:39.382 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:44.100 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:39.439 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:44.101 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:39.440 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:44.105 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:39.394 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:44.113 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:39.452 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:54.076 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:49.414 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:54.078 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:49.416 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:54.092 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:49.430 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:54.118 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:49.407 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:54.123 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:49.412 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:54.126 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:49.464 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:54.130 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:49.469 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:54.136 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:49.425 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:15:54.143 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:49.482 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:04.099 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:59.438 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:04.105 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:59.444 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:04.125 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:48:59.464 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:04.152 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:59.441 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:04.154 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:59.443 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:04.160 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:59.499 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:04.163 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:59.502 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:04.166 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:48:59.455 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:04.181 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:48:59.520 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:14.083 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:49:09.423 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:14.135 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:49:09.474 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:14.150 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:49:09.490 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:14.183 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:49:09.472 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:14.185 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:49:09.474 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:14.186 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:49:09.525 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:14.189 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:49:09.528 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:14.197 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:49:09.487 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:14.202 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:49:09.541 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:24.164 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:49:19.503 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:24.167 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:49:19.506 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:24.187 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:49:19.526 85261== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:24.210 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:49:19.499 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:24.214 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:49:19.503 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:24.222 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:49:19.561 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:24.225 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:49:19.564 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:24.226 INFO:tasks.ceph.mon.b.smithi045.stderr:==00:00:49:19.515 84610== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:24.237 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:49:19.577 85263== Warning: unimplemented fcntl command: 1036 2023-07-29T14:16:29.862 INFO:teuthology.misc:Shutting down mgr daemons... 2023-07-29T14:16:29.863 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-07-29T14:16:29.863 INFO:teuthology.orchestra.run:waiting for 300 2023-07-29T14:16:29.885 INFO:tasks.ceph.mgr.y:Stopped 2023-07-29T14:16:29.886 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-07-29T14:16:29.886 INFO:teuthology.orchestra.run:waiting for 300 2023-07-29T14:16:29.887 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T14:16:29.890+0000 7f403a3f7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i x (PID: 84608) UID: 0 2023-07-29T14:16:29.887 INFO:tasks.ceph.mgr.x.smithi045.stderr:2023-07-29T14:16:29.890+0000 7f403a3f7640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-07-29T14:16:29.916 INFO:tasks.ceph.mgr.x:Stopped 2023-07-29T14:16:29.916 INFO:teuthology.misc:Shutting down mon daemons... 2023-07-29T14:16:29.917 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-07-29T14:16:29.917 INFO:teuthology.orchestra.run:waiting for 300 2023-07-29T14:16:29.940 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-07-29T14:16:29.933+0000 9f30640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 85225) UID: 0 2023-07-29T14:16:29.941 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-07-29T14:16:29.935+0000 9f30640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-29T14:16:30.579 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:49:25.918 85261== 2023-07-29T14:16:30.579 INFO:tasks.ceph.mon.a.smithi040.stderr:==00:00:49:25.918 85261== Exit program on first error (--exit-on-first-error=yes) 2023-07-29T14:16:30.600 INFO:tasks.ceph.mon.a.smithi040.stderr:daemon-helper: command failed with exit status 42 2023-07-29T14:16:30.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-07-29T14:16:36.019 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-07-29T14:16:36.020 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1463, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/misc.py", line 1171, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi040 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2023-07-29T14:16:36.020 INFO:tasks.ceph.mon.a:Stopped 2023-07-29T14:16:36.021 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-07-29T14:16:36.021 INFO:teuthology.orchestra.run:waiting for 300 2023-07-29T14:16:36.037 INFO:tasks.ceph.mon.c.smithi040.stderr:2023-07-29T14:16:36.029+0000 9f30640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 85227) UID: 0 2023-07-29T14:16:36.038 INFO:tasks.ceph.mon.c.smithi040.stderr:2023-07-29T14:16:36.031+0000 9f30640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-07-29T14:16:36.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-07-29T14:16:47.957 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:47.961+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:48.975 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:48.978+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:49.989 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:49.992+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:51.037 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:51.041+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:52.061 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:52.064+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:53.032 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:53.035+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:54.071 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:54.074+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:55.037 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:55.040+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:56.073 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:56.076+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:57.089 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:57.092+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:58.041 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:58.044+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:59.009 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:59.012+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:16:59.982 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:16:59.986+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:01.002 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:01.006+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:02.046 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:02.049+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:03.013 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:03.017+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:04.064 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:04.067+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:05.093 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:05.096+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:06.079 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:06.082+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:07.097 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:07.100+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:08.107 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:08.110+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:09.064 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:09.067+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:10.056 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:10.060+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:11.008 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:11.012+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:11.651 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:17:11.652+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:17:12.009 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:12.013+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:12.989 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:12.992+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:14.019 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:14.023+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:15.019 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:15.021+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:16.066 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:16.069+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:16.651 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:17:16.654+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:17:17.077 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:17.080+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:18.099 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:18.102+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:19.082 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:19.084+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:19.513 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-07-29T14:17:20.042 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:20.045+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:21.042 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:21.045+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:21.652 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:17:21.655+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:17:21.996 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:21.999+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:23.043 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:23.046+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:24.047 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:24.050+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:24.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~5s 2023-07-29T14:17:25.083 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:25.087+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:26.034 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:26.038+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:26.653 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:17:26.656+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 9 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:17:27.082 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:27.086+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:28.098 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:28.102+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:29.102 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:29.106+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:30.119 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~11s 2023-07-29T14:17:30.141 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:30.144+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:31.171 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:31.175+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:31.654 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:17:31.658+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 11 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:17:32.144 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:32.147+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:33.156 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:33.160+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:34.163 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:34.167+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:35.146 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:35.151+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:35.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~16s 2023-07-29T14:17:36.162 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:36.167+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:36.655 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:17:36.660+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 13 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:17:37.138 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:37.143+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:38.093 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:38.098+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:39.052 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:39.056+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:40.066 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:40.070+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:40.726 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~21s 2023-07-29T14:17:41.100 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:41.104+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:41.656 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:17:41.661+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 16 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:17:42.109 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:42.113+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:43.160 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:43.164+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:44.126 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:44.130+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:45.117 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:45.122+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:46.028 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~27s 2023-07-29T14:17:46.084 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:46.088+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:46.657 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:17:46.662+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 19 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:17:47.108 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:47.113+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:48.084 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:48.088+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:49.074 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:49.078+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:50.097 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:50.102+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:51.137 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:51.142+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:51.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~32s 2023-07-29T14:17:51.658 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:17:51.663+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 21 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:17:52.160 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:52.164+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:53.143 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:53.147+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:54.138 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:54.143+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:55.139 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:55.143+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:56.113 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:56.117+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:56.634 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~37s 2023-07-29T14:17:56.660 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:17:56.664+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 24 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:17:57.106 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:57.111+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:58.078 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:58.083+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:17:59.071 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:17:59.076+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:00.089 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:00.093+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:01.059 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:01.064+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:01.660 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:01.665+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 27 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:01.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~42s 2023-07-29T14:18:02.036 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:02.040+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:03.031 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:03.036+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:04.039 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:04.044+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:05.043 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:05.049+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:06.064 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:06.069+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:06.662 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:06.666+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 29 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:07.029 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:07.033+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:07.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~48s 2023-07-29T14:18:08.025 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:08.029+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:09.046 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:09.051+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:10.042 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:10.047+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:11.010 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:11.015+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:11.662 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:11.668+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 32 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:11.974 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:11.979+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:12.543 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~53s 2023-07-29T14:18:12.963 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:12.968+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:13.916 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:13.921+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:14.877 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:14.882+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:15.922 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:15.927+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:16.663 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:16.668+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 35 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:16.897 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:16.901+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:17.846 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~58s 2023-07-29T14:18:17.931 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:17.935+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:18.908 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:18.912+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:19.945 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:19.949+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:20.917 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:20.922+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:21.664 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:21.669+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 37 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:21.948 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:21.953+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:22.984 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:22.989+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:23.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~64s 2023-07-29T14:18:23.965 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:23.969+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:24.922 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:24.927+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:25.874 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:25.879+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:26.665 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:26.670+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 40 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:26.845 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:26.849+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:27.810 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:27.814+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:28.451 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~69s 2023-07-29T14:18:28.817 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:28.822+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:29.781 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:29.786+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:30.733 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:30.738+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:31.667 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:31.671+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 43 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:31.748 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:31.752+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:32.712 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:32.717+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:33.736 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:33.741+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:33.754 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~74s 2023-07-29T14:18:34.708 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:34.713+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:35.707 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:35.712+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:36.667 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:36.673+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 45 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:36.756 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:36.752+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:37.711 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:37.715+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:38.725 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:38.730+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:39.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~80s 2023-07-29T14:18:39.714 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:39.719+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:40.674 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:40.679+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:41.669 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:41.674+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 48 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:41.684 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:41.690+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:42.677 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:42.683+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:43.662 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:43.667+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:44.361 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~85s 2023-07-29T14:18:44.680 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:44.686+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:45.643 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:45.648+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:46.664 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:46.669+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:46.669 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:46.675+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 51 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:47.647 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:47.652+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:48.667 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:48.673+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:49.622 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:49.627+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:49.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~90s 2023-07-29T14:18:50.665 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:50.670+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:51.670 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:51.676+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 53 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:51.685 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:51.690+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:52.694 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:52.699+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:53.711 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:53.717+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:54.752 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:54.757+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:54.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~95s 2023-07-29T14:18:55.767 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:55.772+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:56.671 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:18:56.677+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 56 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:18:56.766 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:56.772+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:57.721 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:57.726+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:58.772 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:58.777+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:18:59.817 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:18:59.822+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:00.268 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~101s 2023-07-29T14:19:00.859 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:00.864+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:01.673 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:01.678+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 59 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:01.874 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:01.879+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:02.848 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:02.854+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:03.893 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:03.899+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:04.893 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:04.899+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:05.570 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~106s 2023-07-29T14:19:05.852 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:05.858+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:06.673 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:06.679+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 61 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:06.836 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:06.841+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:07.872 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:07.877+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:08.873 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:08.879+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:09.902 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:09.908+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:10.873 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~111s 2023-07-29T14:19:10.882 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:10.888+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:11.674 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:11.680+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 64 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:11.878 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:11.884+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:12.907 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:12.912+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:13.859 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:13.865+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:14.847 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:14.853+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:15.805 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:15.811+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:16.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~117s 2023-07-29T14:19:16.675 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:16.681+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 67 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:16.820 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:16.825+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:17.832 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:17.838+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:18.849 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:18.854+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:19.819 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:19.824+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:20.799 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:20.804+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:21.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~122s 2023-07-29T14:19:21.676 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:21.681+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 69 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:21.842 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:21.847+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:22.823 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:22.829+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:23.787 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:23.791+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:24.742 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:24.748+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:25.776 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:25.781+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:26.677 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:26.683+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 72 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:26.767 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:26.771+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:26.781 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~127s 2023-07-29T14:19:27.804 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:27.809+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:28.847 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:28.853+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:29.897 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:29.902+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:30.874 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:30.880+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:31.678 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:31.684+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 75 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:31.840 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:31.845+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:32.084 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~133s 2023-07-29T14:19:32.829 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:32.834+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:33.813 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:33.818+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:34.839 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:34.844+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:35.879 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:35.884+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:36.679 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:36.685+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 77 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:36.920 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:36.925+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:37.387 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~138s 2023-07-29T14:19:37.927 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:37.933+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:38.883 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:38.889+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:39.900 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:39.906+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:40.930 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:40.936+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:41.680 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:41.687+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 80 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:41.913 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:41.919+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:42.689 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~143s 2023-07-29T14:19:42.946 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:42.952+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:43.906 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:43.911+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:44.880 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:44.886+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:45.869 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:45.875+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:46.681 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:46.688+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 83 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:46.887 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:46.894+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:47.887 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:47.893+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:47.992 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~149s 2023-07-29T14:19:48.849 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:48.855+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:49.867 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:49.873+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:50.856 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:50.861+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:51.682 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:51.689+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 85 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:51.886 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:51.892+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:52.935 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:52.942+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:53.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~154s 2023-07-29T14:19:53.967 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:53.973+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:54.968 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:54.974+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:56.011 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:56.017+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:56.683 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:19:56.690+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 88 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:19:56.964 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:56.970+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:57.973 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:57.979+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:19:58.598 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~159s 2023-07-29T14:19:58.984 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:19:58.990+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:00.022 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:00.028+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:01.013 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:01.019+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:01.684 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:01.691+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 91 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:02.050 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:02.056+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:03.086 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:03.092+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:03.901 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~164s 2023-07-29T14:20:04.079 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:04.086+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:05.109 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:05.115+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:06.116 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:06.123+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:06.685 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:06.692+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 93 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:07.110 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:07.116+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:08.072 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:08.078+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:09.101 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:09.108+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:09.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~170s 2023-07-29T14:20:10.147 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:10.154+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:11.114 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:11.121+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:11.686 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:11.693+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 96 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:12.135 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:12.141+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:13.121 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:13.127+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:14.166 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:14.173+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:14.506 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~175s 2023-07-29T14:20:15.186 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:15.192+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:16.237 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:16.244+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:16.687 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:16.694+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 99 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:17.264 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:17.271+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:18.301 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:18.308+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:19.342 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:19.348+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:19.809 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~180s 2023-07-29T14:20:20.359 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:20.366+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:21.324 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:21.330+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:21.688 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:21.694+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 101 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:22.319 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:22.324+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:23.362 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:23.368+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:24.397 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:24.402+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:25.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~186s 2023-07-29T14:20:25.375 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:25.381+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:26.361 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:26.367+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:26.689 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:26.695+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 104 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:27.375 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:27.382+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:28.372 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:28.378+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:29.365 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:29.372+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:30.384 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:30.389+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:30.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~191s 2023-07-29T14:20:31.410 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:31.417+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:31.690 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:31.696+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 111 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:32.386 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:32.392+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:33.404 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:33.410+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:34.413 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:34.419+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:35.423 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:35.429+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:35.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~196s 2023-07-29T14:20:36.415 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:36.422+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:36.691 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:36.697+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 118 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:37.458 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:37.464+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:38.431 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:38.437+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:39.397 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:39.404+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:40.388 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:40.395+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:41.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~202s 2023-07-29T14:20:41.377 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:41.384+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:41.692 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:41.700+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 126 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:42.396 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:42.403+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:43.432 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:43.439+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:44.415 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:44.421+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:45.452 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:45.459+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:46.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~207s 2023-07-29T14:20:46.451 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:46.458+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:46.693 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:46.701+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 134 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:47.490 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:47.497+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:48.462 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:48.469+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:49.486 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:49.493+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:50.477 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:50.484+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:51.508 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:51.516+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:51.630 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~212s 2023-07-29T14:20:51.694 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:51.702+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 141 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:52.519 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:52.525+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:53.520 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:53.527+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:54.483 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:54.490+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:55.496 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:55.503+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:56.527 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:56.534+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:56.696 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:20:56.703+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 149 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:20:56.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~217s 2023-07-29T14:20:57.536 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:57.543+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:58.556 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:58.562+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:20:59.515 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:20:59.522+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:00.486 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:00.493+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:01.470 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:01.477+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:01.697 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:01.704+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 157 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:02.237 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~223s 2023-07-29T14:21:02.498 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:02.505+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:03.484 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:03.491+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:04.503 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:04.511+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:05.536 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:05.543+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:06.565 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:06.572+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:06.699 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:06.706+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 164 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:07.540 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~228s 2023-07-29T14:21:07.556 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:07.562+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:08.594 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:08.601+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:09.623 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:09.630+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:10.606 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:10.613+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:11.570 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:11.578+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:11.700 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:11.707+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 172 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:12.583 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:12.590+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:12.843 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~233s 2023-07-29T14:21:13.587 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:13.595+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:14.592 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:14.599+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:15.557 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:15.564+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:16.539 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:16.546+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:16.701 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:16.708+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 180 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:17.508 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:17.515+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:18.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~239s 2023-07-29T14:21:18.492 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:18.499+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:19.453 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:19.459+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:20.407 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:20.413+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:21.381 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:21.387+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:21.702 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:21.709+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 187 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:22.345 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:22.351+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:23.341 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:23.347+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:23.450 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~244s 2023-07-29T14:21:24.336 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:24.343+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:25.310 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:25.316+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:26.343 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:26.350+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:26.703 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:26.710+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 195 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:27.379 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:27.385+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:28.371 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:28.377+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:28.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~249s 2023-07-29T14:21:29.340 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:29.347+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:30.333 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:30.340+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:31.314 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:31.321+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:31.705 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:31.712+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 203 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:32.313 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:32.320+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:33.363 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:33.369+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:34.058 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~255s 2023-07-29T14:21:34.323 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:34.330+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:35.086 INFO:tasks.ceph:Checking cluster log for badness... 2023-07-29T14:21:35.087 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-07-29T14:21:35.127 INFO:teuthology.orchestra.run.smithi040.stdout:2023-07-29T13:39:28.628828+0000 mon.a (mon.0) 1527 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2023-07-29T14:21:35.128 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-07-29T14:21:35.128 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-07-29T14:21:35.204 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-07-29T14:21:35.280 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-07-29T14:21:35.345 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:35.353+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:35.357 INFO:teuthology.orchestra.run.smithi040.stdout:2023-07-29T13:39:28.628828+0000 mon.a (mon.0) 1527 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2023-07-29T14:21:35.357 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2023-07-29T14:21:35.358 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-07-29T14:21:35.506 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2023-07-29T14:21:35.506 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-07-29T14:21:35.668 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2023-07-29T14:21:35.669 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-07-29T14:21:35.749 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2023-07-29T14:21:35.750 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-07-29T14:21:35.826 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi045.front.sepia.ceph.com 2023-07-29T14:21:35.826 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-07-29T14:21:35.959 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi045.front.sepia.ceph.com 2023-07-29T14:21:35.960 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-07-29T14:21:35.972 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:54:31.316 85263== 2023-07-29T14:21:35.973 INFO:tasks.ceph.mon.c.smithi040.stderr:==00:00:54:31.316 85263== Exit program on first error (--exit-on-first-error=yes) 2023-07-29T14:21:35.992 INFO:tasks.ceph.mon.c.smithi040.stderr:daemon-helper: command failed with exit status 42 2023-07-29T14:21:36.079 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi045.front.sepia.ceph.com 2023-07-29T14:21:36.080 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-07-29T14:21:36.163 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi045.front.sepia.ceph.com 2023-07-29T14:21:36.164 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:36.302 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:36.309+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:umount: /var/lib/ceph/osd/ceph-7: target is busy. 2023-07-29T14:21:36.401 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-29T14:21:36.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo PATH=/usr/sbin:$PATH lsof ; ps auxf 2023-07-29T14:21:36.700 INFO:teuthology.orchestra.run.smithi045.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2023-07-29T14:21:36.700 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.700 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.701 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root txt REG 8,1 102120 16840 /usr/lib/systemd/systemd 2023-07-29T14:21:36.701 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 594568 1228 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-07-29T14:21:36.701 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.701 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.701 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.701 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.701 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 21219 11939 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-07-29T14:21:36.702 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.702 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.702 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.702 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:36.702 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.702 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 28552 9218 /usr/lib64/libattr.so.1.1.2501 2023-07-29T14:21:36.702 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.702 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:36.703 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.703 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.703 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.703 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:36.703 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.703 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 3640680 43416 /usr/lib64/systemd/libsystemd-shared-252.so 2023-07-29T14:21:36.703 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.703 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 41120 9220 /usr/lib64/libacl.so.1.1.2301 2023-07-29T14:21:36.704 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.704 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.704 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 107984 9595 /usr/lib64/libkmod.so.2.3.6 2023-07-29T14:21:36.704 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.704 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:36.704 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 127680 9386 /usr/lib64/libseccomp.so.2.5.2 2023-07-29T14:21:36.704 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.705 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 2112568 43415 /usr/lib64/systemd/libsystemd-core-252.so 2023-07-29T14:21:36.705 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.705 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-07-29T14:21:36.705 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.705 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.705 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.705 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 3w CHR 1,11 0t0 10 /dev/kmsg 2023-07-29T14:21:36.706 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 4u a_inode 0,14 0 2079 [eventpoll:5,6,9,14,15,16,17,18,19,41,42,44,45,52,64,75,81,82,83,84,85,86,87,88,89,90,91,93,94,95,96,123...] 2023-07-29T14:21:36.706 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 5u a_inode 0,14 0 2079 [signalfd] 2023-07-29T14:21:36.706 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 6r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.706 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 7r DIR 0,25 0 1 /sys/fs/cgroup 2023-07-29T14:21:36.706 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 8u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.706 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 9u a_inode 0,14 0 2079 [eventpoll:10,13] 2023-07-29T14:21:36.706 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 10r REG 0,20 0 3979 /proc/1/mountinfo 2023-07-29T14:21:36.706 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 11r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.707 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 12u netlink 0t0 15616 AUDIT 2023-07-29T14:21:36.707 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 13r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.707 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 14r REG 0,20 0 4026532076 /proc/swaps 2023-07-29T14:21:36.707 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 15u netlink 0t0 15608 KOBJECT_UEVENT 2023-07-29T14:21:36.707 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 16u unix 0xffff89b9c6be0440 0t0 15610 /run/systemd/private type=STREAM (LISTEN) 2023-07-29T14:21:36.707 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 17r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.707 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 18u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.707 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 19r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.708 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 21u a_inode 0,14 0 2079 bpf-prog 2023-07-29T14:21:36.708 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 22u a_inode 0,14 0 2079 bpf-prog 2023-07-29T14:21:36.708 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 23u a_inode 0,14 0 2079 bpf-prog 2023-07-29T14:21:36.708 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 24u a_inode 0,14 0 2079 bpf-prog 2023-07-29T14:21:36.708 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 25u unix 0xffff89b9d8225940 0t0 1588 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.708 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 26u a_inode 0,14 0 2079 bpf-prog 2023-07-29T14:21:36.708 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 27r CHR 10,235 0t0 111 /dev/autofs 2023-07-29T14:21:36.709 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 28u a_inode 0,14 0 2079 bpf-prog 2023-07-29T14:21:36.709 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 29u a_inode 0,14 0 2079 bpf-prog 2023-07-29T14:21:36.709 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 30u a_inode 0,14 0 2079 bpf-prog 2023-07-29T14:21:36.709 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 31u a_inode 0,14 0 2079 bpf-prog 2023-07-29T14:21:36.709 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 32u a_inode 0,14 0 2079 bpf-prog 2023-07-29T14:21:36.709 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 41u unix 0xffff89b9c39d7700 0t0 3981 /run/systemd/notify type=DGRAM (CONNECTED) 2023-07-29T14:21:36.709 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 42u unix 0xffff89b9c39d1540 0t0 3982 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.710 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 43u unix 0xffff89b9c39d2640 0t0 3983 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.710 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 44u unix 0xffff89b9c7b88440 0t0 3987 /run/systemd/io.system.ManagedOOM type=STREAM (LISTEN) 2023-07-29T14:21:36.710 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 45u unix 0xffff89b9c7b8c840 0t0 3986 /run/systemd/userdb/io.systemd.DynamicUser type=STREAM (LISTEN) 2023-07-29T14:21:36.710 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 51r FIFO 0,13 0t0 1526 pipe 2023-07-29T14:21:36.710 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 52u unix 0xffff89b9d8219100 0t0 1529 /run/lvm/lvmpolld.socket type=STREAM (LISTEN) 2023-07-29T14:21:36.710 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 53u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.710 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 59u unix 0xffff89b9c39d2ec0 0t0 3994 /run/systemd/journal/dev-log type=DGRAM (CONNECTED) 2023-07-29T14:21:36.710 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 60u unix 0xffff89b9c82f7b40 0t0 20675 type=STREAM (CONNECTED) 2023-07-29T14:21:36.711 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 61u unix 0xffff89b9c82f1980 0t0 20667 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2023-07-29T14:21:36.711 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 64u FIFO 0,24 0t0 444 /run/initctl 2023-07-29T14:21:36.711 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 66u FIFO 0,24 0t0 438 /run/dmeventd-server 2023-07-29T14:21:36.711 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 67u FIFO 0,24 0t0 439 /run/dmeventd-client 2023-07-29T14:21:36.711 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 74u unix 0xffff89b9d8221980 0t0 1543 /run/udev/control type=SEQPACKET (LISTEN) 2023-07-29T14:21:36.712 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 75u unix 0xffff89b9d8224840 0t0 1541 /run/systemd/coredump type=SEQPACKET (LISTEN) 2023-07-29T14:21:36.712 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 76u unix 0xffff89b9c82f1540 0t0 20669 /run/.heim_org.h5l.kcm-socket type=STREAM (LISTEN) 2023-07-29T14:21:36.712 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 81u unix 0xffff89b9c8e7aa80 0t0 2351 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.712 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 82u unix 0xffff89b9c82f6e80 0t0 16630 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.712 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 83u unix 0xffff89b9c6be61c0 0t0 21714 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.712 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 84u unix 0xffff89b9d821a640 0t0 21715 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.712 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 85u unix 0xffff89b9d8219980 0t0 21718 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.713 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 86u unix 0xffff89b9c7b46e80 0t0 21719 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.713 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 87u unix 0xffff89b9c7b47b40 0t0 21720 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.713 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 88u unix 0xffff89b9c82f4840 0t0 21730 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.713 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 89u unix 0xffff89b9d8218880 0t0 16688 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.713 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 90u unix 0xffff89b9c6be0cc0 0t0 16689 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.713 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 91u unix 0xffff89b9c7b41100 0t0 16690 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.713 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 93u unix 0xffff89b9c8e79dc0 0t0 21771 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.713 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 94u unix 0xffff89b9c7b45d80 0t0 16913 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.714 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 95u unix 0xffff89b9d8160000 0t0 22228 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.714 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 96u unix 0xffff89b9c6be4840 0t0 162155 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.714 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 115u netlink 0t0 1545 KOBJECT_UEVENT 2023-07-29T14:21:36.714 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 116u unix 0xffff89b9c39d3740 0t0 3996 /run/systemd/journal/socket type=DGRAM (CONNECTED) 2023-07-29T14:21:36.714 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 117u unix 0xffff89b9c7b8f2c0 0t0 3998 /run/systemd/journal/stdout type=STREAM (LISTEN) 2023-07-29T14:21:36.714 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 118u unix 0xffff89b9d821bb80 0t0 1531 /run/rpcbind.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.714 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 119u IPv4 17734 0t0 TCP *:sunrpc (LISTEN) 2023-07-29T14:21:36.714 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 120u IPv4 17740 0t0 UDP *:sunrpc 2023-07-29T14:21:36.715 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 121u IPv6 17746 0t0 TCP *:sunrpc (LISTEN) 2023-07-29T14:21:36.715 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 122u IPv6 17752 0t0 UDP *:sunrpc 2023-07-29T14:21:36.715 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 1 root 123u CHR 10,242 0t0 165 /dev/rfkill 2023-07-29T14:21:36.715 INFO:teuthology.orchestra.run.smithi045.stdout:kthreadd 2 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.715 INFO:teuthology.orchestra.run.smithi045.stdout:kthreadd 2 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.715 INFO:teuthology.orchestra.run.smithi045.stdout:kthreadd 2 root txt unknown /proc/2/exe 2023-07-29T14:21:36.715 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_gp 3 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.715 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_gp 3 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.716 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_gp 3 root txt unknown /proc/3/exe 2023-07-29T14:21:36.716 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_par_g 4 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.716 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_par_g 4 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.716 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_par_g 4 root txt unknown /proc/4/exe 2023-07-29T14:21:36.716 INFO:teuthology.orchestra.run.smithi045.stdout:slub_flus 5 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.716 INFO:teuthology.orchestra.run.smithi045.stdout:slub_flus 5 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.716 INFO:teuthology.orchestra.run.smithi045.stdout:slub_flus 5 root txt unknown /proc/5/exe 2023-07-29T14:21:36.717 INFO:teuthology.orchestra.run.smithi045.stdout:netns 6 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.717 INFO:teuthology.orchestra.run.smithi045.stdout:netns 6 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.717 INFO:teuthology.orchestra.run.smithi045.stdout:netns 6 root txt unknown /proc/6/exe 2023-07-29T14:21:36.717 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 8 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.717 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 8 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.717 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 8 root txt unknown /proc/8/exe 2023-07-29T14:21:36.717 INFO:teuthology.orchestra.run.smithi045.stdout:mm_percpu 10 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.718 INFO:teuthology.orchestra.run.smithi045.stdout:mm_percpu 10 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.718 INFO:teuthology.orchestra.run.smithi045.stdout:mm_percpu 10 root txt unknown /proc/10/exe 2023-07-29T14:21:36.718 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_tasks 12 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.718 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_tasks 12 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.718 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_tasks 12 root txt unknown /proc/12/exe 2023-07-29T14:21:36.718 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_tasks 13 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.718 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_tasks 13 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.718 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_tasks 13 root txt unknown /proc/13/exe 2023-07-29T14:21:36.719 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_tasks 14 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.719 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_tasks 14 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.719 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_tasks 14 root txt unknown /proc/14/exe 2023-07-29T14:21:36.719 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 15 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.719 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 15 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.719 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 15 root txt unknown /proc/15/exe 2023-07-29T14:21:36.719 INFO:teuthology.orchestra.run.smithi045.stdout:pr/ttyS1 16 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.719 INFO:teuthology.orchestra.run.smithi045.stdout:pr/ttyS1 16 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.720 INFO:teuthology.orchestra.run.smithi045.stdout:pr/ttyS1 16 root txt unknown /proc/16/exe 2023-07-29T14:21:36.720 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_preem 17 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.720 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_preem 17 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.720 INFO:teuthology.orchestra.run.smithi045.stdout:rcu_preem 17 root txt unknown /proc/17/exe 2023-07-29T14:21:36.720 INFO:teuthology.orchestra.run.smithi045.stdout:migration 18 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.720 INFO:teuthology.orchestra.run.smithi045.stdout:migration 18 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.720 INFO:teuthology.orchestra.run.smithi045.stdout:migration 18 root txt unknown /proc/18/exe 2023-07-29T14:21:36.721 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 19 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.721 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 19 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.721 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 19 root txt unknown /proc/19/exe 2023-07-29T14:21:36.721 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/0 21 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.721 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/0 21 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.721 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/0 21 root txt unknown /proc/21/exe 2023-07-29T14:21:36.721 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/1 22 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.721 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/1 22 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.722 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/1 22 root txt unknown /proc/22/exe 2023-07-29T14:21:36.722 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 23 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.722 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 23 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.722 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 23 root txt unknown /proc/23/exe 2023-07-29T14:21:36.722 INFO:teuthology.orchestra.run.smithi045.stdout:migration 24 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.722 INFO:teuthology.orchestra.run.smithi045.stdout:migration 24 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.722 INFO:teuthology.orchestra.run.smithi045.stdout:migration 24 root txt unknown /proc/24/exe 2023-07-29T14:21:36.722 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 25 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.723 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 25 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.723 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 25 root txt unknown /proc/25/exe 2023-07-29T14:21:36.723 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 27 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.723 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 27 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.723 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 27 root txt unknown /proc/27/exe 2023-07-29T14:21:36.723 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/2 28 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.723 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/2 28 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.723 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/2 28 root txt unknown /proc/28/exe 2023-07-29T14:21:36.724 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 29 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.724 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 29 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.724 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 29 root txt unknown /proc/29/exe 2023-07-29T14:21:36.724 INFO:teuthology.orchestra.run.smithi045.stdout:migration 30 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.724 INFO:teuthology.orchestra.run.smithi045.stdout:migration 30 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.724 INFO:teuthology.orchestra.run.smithi045.stdout:migration 30 root txt unknown /proc/30/exe 2023-07-29T14:21:36.724 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 31 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.724 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 31 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.725 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 31 root txt unknown /proc/31/exe 2023-07-29T14:21:36.725 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 33 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.725 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 33 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.725 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 33 root txt unknown /proc/33/exe 2023-07-29T14:21:36.725 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/3 34 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.725 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/3 34 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.725 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/3 34 root txt unknown /proc/34/exe 2023-07-29T14:21:36.725 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 35 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.726 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 35 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.726 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 35 root txt unknown /proc/35/exe 2023-07-29T14:21:36.726 INFO:teuthology.orchestra.run.smithi045.stdout:migration 36 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.726 INFO:teuthology.orchestra.run.smithi045.stdout:migration 36 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.726 INFO:teuthology.orchestra.run.smithi045.stdout:migration 36 root txt unknown /proc/36/exe 2023-07-29T14:21:36.726 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 37 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.726 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 37 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.727 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 37 root txt unknown /proc/37/exe 2023-07-29T14:21:36.727 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 39 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.727 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 39 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.727 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 39 root txt unknown /proc/39/exe 2023-07-29T14:21:36.727 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/4 40 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.727 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/4 40 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.727 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/4 40 root txt unknown /proc/40/exe 2023-07-29T14:21:36.727 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 41 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.728 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 41 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.728 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 41 root txt unknown /proc/41/exe 2023-07-29T14:21:36.728 INFO:teuthology.orchestra.run.smithi045.stdout:migration 42 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.728 INFO:teuthology.orchestra.run.smithi045.stdout:migration 42 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.728 INFO:teuthology.orchestra.run.smithi045.stdout:migration 42 root txt unknown /proc/42/exe 2023-07-29T14:21:36.728 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 43 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.728 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 43 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.729 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 43 root txt unknown /proc/43/exe 2023-07-29T14:21:36.729 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 44 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.729 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 44 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.729 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 44 root txt unknown /proc/44/exe 2023-07-29T14:21:36.729 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 45 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.729 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 45 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.729 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 45 root txt unknown /proc/45/exe 2023-07-29T14:21:36.729 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/5 46 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.730 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/5 46 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.730 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/5 46 root txt unknown /proc/46/exe 2023-07-29T14:21:36.730 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 47 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.730 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 47 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.730 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 47 root txt unknown /proc/47/exe 2023-07-29T14:21:36.730 INFO:teuthology.orchestra.run.smithi045.stdout:migration 48 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.730 INFO:teuthology.orchestra.run.smithi045.stdout:migration 48 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.730 INFO:teuthology.orchestra.run.smithi045.stdout:migration 48 root txt unknown /proc/48/exe 2023-07-29T14:21:36.731 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 49 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.731 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 49 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.731 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 49 root txt unknown /proc/49/exe 2023-07-29T14:21:36.731 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 51 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.731 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 51 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.731 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 51 root txt unknown /proc/51/exe 2023-07-29T14:21:36.731 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/6 52 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.731 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/6 52 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.732 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/6 52 root txt unknown /proc/52/exe 2023-07-29T14:21:36.732 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 53 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.732 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 53 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.732 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 53 root txt unknown /proc/53/exe 2023-07-29T14:21:36.732 INFO:teuthology.orchestra.run.smithi045.stdout:migration 54 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.732 INFO:teuthology.orchestra.run.smithi045.stdout:migration 54 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.732 INFO:teuthology.orchestra.run.smithi045.stdout:migration 54 root txt unknown /proc/54/exe 2023-07-29T14:21:36.732 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 55 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.733 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 55 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.733 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 55 root txt unknown /proc/55/exe 2023-07-29T14:21:36.733 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 57 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.733 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 57 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.733 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 57 root txt unknown /proc/57/exe 2023-07-29T14:21:36.733 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/7 58 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.733 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/7 58 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.733 INFO:teuthology.orchestra.run.smithi045.stdout:cpuhp/7 58 root txt unknown /proc/58/exe 2023-07-29T14:21:36.734 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 59 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.734 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 59 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.734 INFO:teuthology.orchestra.run.smithi045.stdout:idle_inje 59 root txt unknown /proc/59/exe 2023-07-29T14:21:36.734 INFO:teuthology.orchestra.run.smithi045.stdout:migration 60 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.734 INFO:teuthology.orchestra.run.smithi045.stdout:migration 60 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.734 INFO:teuthology.orchestra.run.smithi045.stdout:migration 60 root txt unknown /proc/60/exe 2023-07-29T14:21:36.734 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 61 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.734 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 61 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.735 INFO:teuthology.orchestra.run.smithi045.stdout:ksoftirqd 61 root txt unknown /proc/61/exe 2023-07-29T14:21:36.735 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 63 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.735 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 63 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.735 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 63 root txt unknown /proc/63/exe 2023-07-29T14:21:36.735 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 67 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.735 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 67 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.735 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 67 root txt unknown /proc/67/exe 2023-07-29T14:21:36.736 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 70 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.736 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 70 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.736 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 70 root txt unknown /proc/70/exe 2023-07-29T14:21:36.736 INFO:teuthology.orchestra.run.smithi045.stdout:kdevtmpfs 71 root cwd DIR 0,5 3500 1 / 2023-07-29T14:21:36.736 INFO:teuthology.orchestra.run.smithi045.stdout:kdevtmpfs 71 root rtd DIR 0,5 3500 1 / 2023-07-29T14:21:36.736 INFO:teuthology.orchestra.run.smithi045.stdout:kdevtmpfs 71 root txt unknown /proc/71/exe 2023-07-29T14:21:36.736 INFO:teuthology.orchestra.run.smithi045.stdout:inet_frag 72 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.736 INFO:teuthology.orchestra.run.smithi045.stdout:inet_frag 72 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.737 INFO:teuthology.orchestra.run.smithi045.stdout:inet_frag 72 root txt unknown /proc/72/exe 2023-07-29T14:21:36.737 INFO:teuthology.orchestra.run.smithi045.stdout:kauditd 73 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.737 INFO:teuthology.orchestra.run.smithi045.stdout:kauditd 73 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.737 INFO:teuthology.orchestra.run.smithi045.stdout:kauditd 73 root txt unknown /proc/73/exe 2023-07-29T14:21:36.737 INFO:teuthology.orchestra.run.smithi045.stdout:khungtask 75 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.737 INFO:teuthology.orchestra.run.smithi045.stdout:khungtask 75 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.737 INFO:teuthology.orchestra.run.smithi045.stdout:khungtask 75 root txt unknown /proc/75/exe 2023-07-29T14:21:36.737 INFO:teuthology.orchestra.run.smithi045.stdout:oom_reape 76 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.738 INFO:teuthology.orchestra.run.smithi045.stdout:oom_reape 76 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.738 INFO:teuthology.orchestra.run.smithi045.stdout:oom_reape 76 root txt unknown /proc/76/exe 2023-07-29T14:21:36.738 INFO:teuthology.orchestra.run.smithi045.stdout:writeback 77 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.738 INFO:teuthology.orchestra.run.smithi045.stdout:writeback 77 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.738 INFO:teuthology.orchestra.run.smithi045.stdout:writeback 77 root txt unknown /proc/77/exe 2023-07-29T14:21:36.738 INFO:teuthology.orchestra.run.smithi045.stdout:kcompactd 78 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.738 INFO:teuthology.orchestra.run.smithi045.stdout:kcompactd 78 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.739 INFO:teuthology.orchestra.run.smithi045.stdout:kcompactd 78 root txt unknown /proc/78/exe 2023-07-29T14:21:36.739 INFO:teuthology.orchestra.run.smithi045.stdout:ksmd 79 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.739 INFO:teuthology.orchestra.run.smithi045.stdout:ksmd 79 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.739 INFO:teuthology.orchestra.run.smithi045.stdout:ksmd 79 root txt unknown /proc/79/exe 2023-07-29T14:21:36.739 INFO:teuthology.orchestra.run.smithi045.stdout:khugepage 80 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.739 INFO:teuthology.orchestra.run.smithi045.stdout:khugepage 80 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.739 INFO:teuthology.orchestra.run.smithi045.stdout:khugepage 80 root txt unknown /proc/80/exe 2023-07-29T14:21:36.739 INFO:teuthology.orchestra.run.smithi045.stdout:cryptd 81 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.740 INFO:teuthology.orchestra.run.smithi045.stdout:cryptd 81 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.740 INFO:teuthology.orchestra.run.smithi045.stdout:cryptd 81 root txt unknown /proc/81/exe 2023-07-29T14:21:36.740 INFO:teuthology.orchestra.run.smithi045.stdout:kintegrit 82 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.740 INFO:teuthology.orchestra.run.smithi045.stdout:kintegrit 82 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.740 INFO:teuthology.orchestra.run.smithi045.stdout:kintegrit 82 root txt unknown /proc/82/exe 2023-07-29T14:21:36.740 INFO:teuthology.orchestra.run.smithi045.stdout:kblockd 83 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.740 INFO:teuthology.orchestra.run.smithi045.stdout:kblockd 83 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.740 INFO:teuthology.orchestra.run.smithi045.stdout:kblockd 83 root txt unknown /proc/83/exe 2023-07-29T14:21:36.741 INFO:teuthology.orchestra.run.smithi045.stdout:blkcg_pun 84 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.741 INFO:teuthology.orchestra.run.smithi045.stdout:blkcg_pun 84 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.741 INFO:teuthology.orchestra.run.smithi045.stdout:blkcg_pun 84 root txt unknown /proc/84/exe 2023-07-29T14:21:36.741 INFO:teuthology.orchestra.run.smithi045.stdout:tpm_dev_w 86 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.741 INFO:teuthology.orchestra.run.smithi045.stdout:tpm_dev_w 86 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.741 INFO:teuthology.orchestra.run.smithi045.stdout:tpm_dev_w 86 root txt unknown /proc/86/exe 2023-07-29T14:21:36.741 INFO:teuthology.orchestra.run.smithi045.stdout:md 87 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.741 INFO:teuthology.orchestra.run.smithi045.stdout:md 87 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.742 INFO:teuthology.orchestra.run.smithi045.stdout:md 87 root txt unknown /proc/87/exe 2023-07-29T14:21:36.742 INFO:teuthology.orchestra.run.smithi045.stdout:edac-poll 88 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.742 INFO:teuthology.orchestra.run.smithi045.stdout:edac-poll 88 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.742 INFO:teuthology.orchestra.run.smithi045.stdout:edac-poll 88 root txt unknown /proc/88/exe 2023-07-29T14:21:36.742 INFO:teuthology.orchestra.run.smithi045.stdout:watchdogd 89 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.742 INFO:teuthology.orchestra.run.smithi045.stdout:watchdogd 89 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.742 INFO:teuthology.orchestra.run.smithi045.stdout:watchdogd 89 root txt unknown /proc/89/exe 2023-07-29T14:21:36.743 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 90 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.743 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 90 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.743 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 90 root txt unknown /proc/90/exe 2023-07-29T14:21:36.743 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 91 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.743 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 91 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.743 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 91 root txt unknown /proc/91/exe 2023-07-29T14:21:36.743 INFO:teuthology.orchestra.run.smithi045.stdout:kswapd0 93 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.743 INFO:teuthology.orchestra.run.smithi045.stdout:kswapd0 93 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.744 INFO:teuthology.orchestra.run.smithi045.stdout:kswapd0 93 root txt unknown /proc/93/exe 2023-07-29T14:21:36.744 INFO:teuthology.orchestra.run.smithi045.stdout:kthrotld 99 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.744 INFO:teuthology.orchestra.run.smithi045.stdout:kthrotld 99 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.744 INFO:teuthology.orchestra.run.smithi045.stdout:kthrotld 99 root txt unknown /proc/99/exe 2023-07-29T14:21:36.744 INFO:teuthology.orchestra.run.smithi045.stdout:acpi_ther 108 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.744 INFO:teuthology.orchestra.run.smithi045.stdout:acpi_ther 108 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.744 INFO:teuthology.orchestra.run.smithi045.stdout:acpi_ther 108 root txt unknown /proc/108/exe 2023-07-29T14:21:36.744 INFO:teuthology.orchestra.run.smithi045.stdout:kmpath_rd 109 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:kmpath_rd 109 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:kmpath_rd 109 root txt unknown /proc/109/exe 2023-07-29T14:21:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:kaluad 110 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:kaluad 110 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:kaluad 110 root txt unknown /proc/110/exe 2023-07-29T14:21:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:mld 111 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:mld 111 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:mld 111 root txt unknown /proc/111/exe 2023-07-29T14:21:36.746 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 112 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.746 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 112 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.746 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 112 root txt unknown /proc/112/exe 2023-07-29T14:21:36.746 INFO:teuthology.orchestra.run.smithi045.stdout:ipv6_addr 113 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.746 INFO:teuthology.orchestra.run.smithi045.stdout:ipv6_addr 113 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.746 INFO:teuthology.orchestra.run.smithi045.stdout:ipv6_addr 113 root txt unknown /proc/113/exe 2023-07-29T14:21:36.746 INFO:teuthology.orchestra.run.smithi045.stdout:kstrp 122 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.747 INFO:teuthology.orchestra.run.smithi045.stdout:kstrp 122 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.747 INFO:teuthology.orchestra.run.smithi045.stdout:kstrp 122 root txt unknown /proc/122/exe 2023-07-29T14:21:36.747 INFO:teuthology.orchestra.run.smithi045.stdout:zswap-shr 136 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.747 INFO:teuthology.orchestra.run.smithi045.stdout:zswap-shr 136 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.747 INFO:teuthology.orchestra.run.smithi045.stdout:zswap-shr 136 root txt unknown /proc/136/exe 2023-07-29T14:21:36.747 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 250 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.747 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 250 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.747 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 250 root txt unknown /proc/250/exe 2023-07-29T14:21:36.748 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 300 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.748 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 300 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.748 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 300 root txt unknown /proc/300/exe 2023-07-29T14:21:36.748 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 303 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.748 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 303 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.748 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 303 root txt unknown /proc/303/exe 2023-07-29T14:21:36.748 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 349 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.748 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 349 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.749 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 349 root txt unknown /proc/349/exe 2023-07-29T14:21:36.749 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 365 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.749 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 365 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.749 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 365 root txt unknown /proc/365/exe 2023-07-29T14:21:36.749 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 388 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.749 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 388 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.749 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 388 root txt unknown /proc/388/exe 2023-07-29T14:21:36.749 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 422 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.750 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 422 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.750 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 422 root txt unknown /proc/422/exe 2023-07-29T14:21:36.750 INFO:teuthology.orchestra.run.smithi045.stdout:ata_sff 440 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.750 INFO:teuthology.orchestra.run.smithi045.stdout:ata_sff 440 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.750 INFO:teuthology.orchestra.run.smithi045.stdout:ata_sff 440 root txt unknown /proc/440/exe 2023-07-29T14:21:36.750 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-wq 441 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.750 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-wq 441 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.750 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-wq 441 root txt unknown /proc/441/exe 2023-07-29T14:21:36.751 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-rese 443 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.751 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-rese 443 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.751 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-rese 443 root txt unknown /proc/443/exe 2023-07-29T14:21:36.751 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-dele 444 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.751 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-dele 444 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.751 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-dele 444 root txt unknown /proc/444/exe 2023-07-29T14:21:36.751 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-auth 445 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.751 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-auth 445 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.752 INFO:teuthology.orchestra.run.smithi045.stdout:nvme-auth 445 root txt unknown /proc/445/exe 2023-07-29T14:21:36.752 INFO:teuthology.orchestra.run.smithi045.stdout:ixgbe 451 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.752 INFO:teuthology.orchestra.run.smithi045.stdout:ixgbe 451 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.752 INFO:teuthology.orchestra.run.smithi045.stdout:ixgbe 451 root txt unknown /proc/451/exe 2023-07-29T14:21:36.752 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_0 454 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.752 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_0 454 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.752 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_0 454 root txt unknown /proc/454/exe 2023-07-29T14:21:36.753 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 455 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.753 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 455 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.753 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 455 root txt unknown /proc/455/exe 2023-07-29T14:21:36.753 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_1 456 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.753 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_1 456 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.753 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_1 456 root txt unknown /proc/456/exe 2023-07-29T14:21:36.753 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 457 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.753 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 457 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.754 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 457 root txt unknown /proc/457/exe 2023-07-29T14:21:36.754 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_2 458 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.754 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_2 458 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.754 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_2 458 root txt unknown /proc/458/exe 2023-07-29T14:21:36.754 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 459 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.754 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 459 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.754 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 459 root txt unknown /proc/459/exe 2023-07-29T14:21:36.754 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_3 460 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.755 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_3 460 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.755 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_3 460 root txt unknown /proc/460/exe 2023-07-29T14:21:36.755 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 461 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.755 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 461 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.755 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 461 root txt unknown /proc/461/exe 2023-07-29T14:21:36.755 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_4 466 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.755 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_4 466 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.755 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_4 466 root txt unknown /proc/466/exe 2023-07-29T14:21:36.756 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 467 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.756 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 467 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.756 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 467 root txt unknown /proc/467/exe 2023-07-29T14:21:36.756 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_5 468 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.756 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_5 468 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.756 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_5 468 root txt unknown /proc/468/exe 2023-07-29T14:21:36.756 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 469 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.757 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 469 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.757 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 469 root txt unknown /proc/469/exe 2023-07-29T14:21:36.757 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_6 470 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.757 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_6 470 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.757 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_6 470 root txt unknown /proc/470/exe 2023-07-29T14:21:36.757 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 471 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.757 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 471 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.757 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 471 root txt unknown /proc/471/exe 2023-07-29T14:21:36.758 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_7 472 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.758 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_7 472 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.758 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_7 472 root txt unknown /proc/472/exe 2023-07-29T14:21:36.758 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 473 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.758 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 473 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.758 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 473 root txt unknown /proc/473/exe 2023-07-29T14:21:36.758 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_8 474 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.759 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_8 474 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.759 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_8 474 root txt unknown /proc/474/exe 2023-07-29T14:21:36.759 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 475 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.759 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 475 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.759 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 475 root txt unknown /proc/475/exe 2023-07-29T14:21:36.759 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_9 476 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.759 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_9 476 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.759 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_eh_9 476 root txt unknown /proc/476/exe 2023-07-29T14:21:36.760 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 477 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.760 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 477 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.760 INFO:teuthology.orchestra.run.smithi045.stdout:scsi_tmf_ 477 root txt unknown /proc/477/exe 2023-07-29T14:21:36.760 INFO:teuthology.orchestra.run.smithi045.stdout:jbd2/sda1 505 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.760 INFO:teuthology.orchestra.run.smithi045.stdout:jbd2/sda1 505 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.760 INFO:teuthology.orchestra.run.smithi045.stdout:jbd2/sda1 505 root txt unknown /proc/505/exe 2023-07-29T14:21:36.760 INFO:teuthology.orchestra.run.smithi045.stdout:ext4-rsv- 506 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.760 INFO:teuthology.orchestra.run.smithi045.stdout:ext4-rsv- 506 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.761 INFO:teuthology.orchestra.run.smithi045.stdout:ext4-rsv- 506 root txt unknown /proc/506/exe 2023-07-29T14:21:36.761 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.761 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.761 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root txt REG 8,1 185480 16847 /usr/lib/systemd/systemd-journald 2023-07-29T14:21:36.761 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 0,24 33554432 58 /run/log/journal/e13cba1d25f14c51a1d04da8db823a4e/system.journal 2023-07-29T14:21:36.761 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.761 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.761 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.762 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.762 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.762 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.762 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:36.762 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.762 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.762 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.762 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 127680 9386 /usr/lib64/libseccomp.so.2.5.2 2023-07-29T14:21:36.763 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:36.763 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.763 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.763 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 28552 9218 /usr/lib64/libattr.so.1.1.2501 2023-07-29T14:21:36.763 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.763 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.763 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.763 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:36.764 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 3640680 43416 /usr/lib64/systemd/libsystemd-shared-252.so 2023-07-29T14:21:36.764 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:36.764 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 107984 9595 /usr/lib64/libkmod.so.2.3.6 2023-07-29T14:21:36.764 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.764 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.764 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 41120 9220 /usr/lib64/libacl.so.1.1.2301 2023-07-29T14:21:36.764 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.765 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.765 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.765 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root mem REG 0,24 8 55 /run/systemd/journal/kernel-seqnum 2023-07-29T14:21:36.765 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.765 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 1w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.765 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 2w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.765 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 3u unix 0xffff89b9c39d2ec0 0t0 3994 /run/systemd/journal/dev-log type=DGRAM (CONNECTED) 2023-07-29T14:21:36.765 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 4u unix 0xffff89b9c39d3740 0t0 3996 /run/systemd/journal/socket type=DGRAM (CONNECTED) 2023-07-29T14:21:36.766 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 5u unix 0xffff89b9c7b8f2c0 0t0 3998 /run/systemd/journal/stdout type=STREAM (LISTEN) 2023-07-29T14:21:36.766 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 6w CHR 1,11 0t0 10 /dev/kmsg 2023-07-29T14:21:36.766 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 7u a_inode 0,14 0 2079 [eventpoll:3,4,5,8,9,10,11,12,13,15,17,18,19,20,21,22,23,24,25,26,27,28,29,31,34] 2023-07-29T14:21:36.766 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 8u CHR 1,11 0t0 10 /dev/kmsg 2023-07-29T14:21:36.766 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 9u unix 0xffff89b9c7b46600 0t0 19507 /run/systemd/journal/io.systemd.journal type=STREAM (LISTEN) 2023-07-29T14:21:36.766 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 10r REG 0,20 0 15300 /proc/sys/kernel/hostname 2023-07-29T14:21:36.766 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 11u a_inode 0,14 0 2079 [signalfd] 2023-07-29T14:21:36.766 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 12u a_inode 0,14 0 2079 [signalfd] 2023-07-29T14:21:36.767 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 13u a_inode 0,14 0 2079 [signalfd] 2023-07-29T14:21:36.767 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 14u unix 0xffff89b9c39b3740 0t0 19508 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.767 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 15u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.767 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 16u REG 0,24 33554432 58 /run/log/journal/e13cba1d25f14c51a1d04da8db823a4e/system.journal 2023-07-29T14:21:36.767 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 17u unix 0xffff89b9c6be61c0 0t0 21714 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.767 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 18u unix 0xffff89b9d821a640 0t0 21715 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.767 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 19u unix 0xffff89b9c82f6e80 0t0 16630 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.767 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 20u unix 0xffff89b9c8e7aa80 0t0 2351 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.768 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 21u unix 0xffff89b9d8160000 0t0 22228 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.768 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 22u unix 0xffff89b9d8218880 0t0 16688 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.768 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 23u unix 0xffff89b9d8219980 0t0 21718 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.768 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 24u unix 0xffff89b9c7b46e80 0t0 21719 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.768 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 25u unix 0xffff89b9c7b47b40 0t0 21720 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.768 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 26u unix 0xffff89b9c6be0cc0 0t0 16689 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.768 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 27u unix 0xffff89b9c7b41100 0t0 16690 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.769 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 28u unix 0xffff89b9c82f4840 0t0 21730 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.769 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 29u unix 0xffff89b9c6be4840 0t0 162155 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.769 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 31u unix 0xffff89b9c8e79dc0 0t0 21771 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.769 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-j 575 root 34u unix 0xffff89b9c7b45d80 0t0 16913 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-07-29T14:21:36.769 INFO:teuthology.orchestra.run.smithi045.stdout:ext4lazyi 581 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.769 INFO:teuthology.orchestra.run.smithi045.stdout:ext4lazyi 581 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.769 INFO:teuthology.orchestra.run.smithi045.stdout:ext4lazyi 581 root txt unknown /proc/581/exe 2023-07-29T14:21:36.769 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.770 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.770 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root txt REG 8,1 589464 11010 /usr/bin/udevadm 2023-07-29T14:21:36.770 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 11730661 54360 /etc/udev/hwdb.bin 2023-07-29T14:21:36.770 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 594568 1228 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-07-29T14:21:36.770 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 21219 11939 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-07-29T14:21:36.770 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 563049 395438 /usr/lib/modules/5.14.0-344.el9.x86_64/modules.symbols.bin 2023-07-29T14:21:36.770 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 818849 395435 /usr/lib/modules/5.14.0-344.el9.x86_64/modules.alias.bin 2023-07-29T14:21:36.770 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 379715 395444 /usr/lib/modules/5.14.0-344.el9.x86_64/modules.dep.bin 2023-07-29T14:21:36.771 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.771 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.771 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:36.771 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.771 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.771 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.771 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.771 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.772 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.772 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.772 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.772 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.772 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 127680 9386 /usr/lib64/libseccomp.so.2.5.2 2023-07-29T14:21:36.772 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:36.772 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:36.773 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.773 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.773 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 28552 9218 /usr/lib64/libattr.so.1.1.2501 2023-07-29T14:21:36.773 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.773 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.773 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:36.773 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.773 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 41120 9220 /usr/lib64/libacl.so.1.1.2301 2023-07-29T14:21:36.774 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.774 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 3640680 43416 /usr/lib64/systemd/libsystemd-shared-252.so 2023-07-29T14:21:36.774 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.774 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 107984 9595 /usr/lib64/libkmod.so.2.3.6 2023-07-29T14:21:36.774 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 11730 395439 /usr/lib/modules/5.14.0-344.el9.x86_64/modules.builtin.bin 2023-07-29T14:21:36.774 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.774 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-07-29T14:21:36.774 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.775 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 1u unix 0xffff89b9c8e78440 0t0 16525 type=STREAM (CONNECTED) 2023-07-29T14:21:36.775 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 2u unix 0xffff89b9c8e78440 0t0 16525 type=STREAM (CONNECTED) 2023-07-29T14:21:36.775 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 3u netlink 0t0 1545 KOBJECT_UEVENT 2023-07-29T14:21:36.775 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 4u unix 0xffff89b9d8221980 0t0 1543 /run/udev/control type=SEQPACKET (LISTEN) 2023-07-29T14:21:36.775 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 5u unix 0xffff89b9c8e83fc0 0t0 16537 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.775 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 6u unix 0xffff89b9c8e81dc0 0t0 16541 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.775 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 7u unix 0xffff89b9c8e872c0 0t0 16542 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.775 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.776 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 9u a_inode 0,14 0 2079 [eventpoll:3,4,6,8,10,11,14] 2023-07-29T14:21:36.776 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 10u a_inode 0,14 0 2079 [signalfd] 2023-07-29T14:21:36.776 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 11u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.776 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 12r REG 8,1 11730661 54360 /etc/udev/hwdb.bin 2023-07-29T14:21:36.776 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-u 594 root 14u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.776 INFO:teuthology.orchestra.run.smithi045.stdout:ipmi-msgh 645 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.776 INFO:teuthology.orchestra.run.smithi045.stdout:ipmi-msgh 645 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.777 INFO:teuthology.orchestra.run.smithi045.stdout:ipmi-msgh 645 root txt unknown /proc/645/exe 2023-07-29T14:21:36.777 INFO:teuthology.orchestra.run.smithi045.stdout:kipmi0 654 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.777 INFO:teuthology.orchestra.run.smithi045.stdout:kipmi0 654 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.777 INFO:teuthology.orchestra.run.smithi045.stdout:kipmi0 654 root txt unknown /proc/654/exe 2023-07-29T14:21:36.777 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.777 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.777 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc txt REG 8,1 61200 2298 /usr/bin/rpcbind 2023-07-29T14:21:36.777 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.778 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.778 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.778 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.778 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.778 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.778 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.778 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.778 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.779 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.779 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.779 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.779 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.779 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.779 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.779 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.779 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.780 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.780 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.780 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.780 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 196120 10846 /usr/lib64/libtirpc.so.3.0.0 2023-07-29T14:21:36.780 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.780 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.780 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc 1u unix 0xffff89b9c82f2640 0t0 20620 type=STREAM (CONNECTED) 2023-07-29T14:21:36.781 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc 2u unix 0xffff89b9c82f2640 0t0 20620 type=STREAM (CONNECTED) 2023-07-29T14:21:36.781 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc 3u unix 0xffff89b9d821bb80 0t0 1531 /run/rpcbind.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.781 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc 4u IPv4 17734 0t0 TCP *:sunrpc (LISTEN) 2023-07-29T14:21:36.781 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc 5u IPv4 17740 0t0 UDP *:sunrpc 2023-07-29T14:21:36.781 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc 6u IPv6 17746 0t0 TCP *:sunrpc (LISTEN) 2023-07-29T14:21:36.781 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc 7u IPv6 17752 0t0 UDP *:sunrpc 2023-07-29T14:21:36.781 INFO:teuthology.orchestra.run.smithi045.stdout:rpcbind 669 rpc 8rW REG 0,24 0 1204 /run/rpcbind/rpcbind.lock 2023-07-29T14:21:36.781 INFO:teuthology.orchestra.run.smithi045.stdout:rpciod 670 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.782 INFO:teuthology.orchestra.run.smithi045.stdout:rpciod 670 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.782 INFO:teuthology.orchestra.run.smithi045.stdout:rpciod 670 root txt unknown /proc/670/exe 2023-07-29T14:21:36.782 INFO:teuthology.orchestra.run.smithi045.stdout:xprtiod 671 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.782 INFO:teuthology.orchestra.run.smithi045.stdout:xprtiod 671 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.782 INFO:teuthology.orchestra.run.smithi045.stdout:xprtiod 671 root txt unknown /proc/671/exe 2023-07-29T14:21:36.782 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.782 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.783 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root txt REG 8,1 145776 11191 /usr/sbin/auditd 2023-07-29T14:21:36.783 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.783 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.783 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.783 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.783 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.783 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.783 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 144728 9145 /usr/lib64/libauparse.so.0.0.0 2023-07-29T14:21:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root 3u netlink 0t0 22766 AUDIT 2023-07-29T14:21:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root 4w REG 8,1 5499631 59420 /var/log/audit/audit.log 2023-07-29T14:21:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root 6u unix 0xffff89b9c8e80cc0 0t0 16640 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root 7u unix 0xffff89b9c8e7e600 0t0 16639 type=STREAM (CONNECTED) 2023-07-29T14:21:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root 8u unix 0xffff89b9c8e7d940 0t0 16641 type=STREAM (CONNECTED) 2023-07-29T14:21:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root 9u unix 0xffff89b9c8e7dd80 0t0 16642 type=STREAM (CONNECTED) 2023-07-29T14:21:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 root 10u a_inode 0,14 0 2079 [eventfd:0] 2023-07-29T14:21:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root txt REG 8,1 145776 11191 /usr/sbin/auditd 2023-07-29T14:21:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 144728 9145 /usr/lib64/libauparse.so.0.0.0 2023-07-29T14:21:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root 3u netlink 0t0 22766 AUDIT 2023-07-29T14:21:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root 4w REG 8,1 5499631 59420 /var/log/audit/audit.log 2023-07-29T14:21:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root 6u unix 0xffff89b9c8e80cc0 0t0 16640 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root 7u unix 0xffff89b9c8e7e600 0t0 16639 type=STREAM (CONNECTED) 2023-07-29T14:21:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root 8u unix 0xffff89b9c8e7d940 0t0 16641 type=STREAM (CONNECTED) 2023-07-29T14:21:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root 9u unix 0xffff89b9c8e7dd80 0t0 16642 type=STREAM (CONNECTED) 2023-07-29T14:21:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 673 auditd root 10u a_inode 0,14 0 2079 [eventfd:0] 2023-07-29T14:21:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root txt REG 8,1 145776 11191 /usr/sbin/auditd 2023-07-29T14:21:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 144728 9145 /usr/lib64/libauparse.so.0.0.0 2023-07-29T14:21:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root 3u netlink 0t0 22766 AUDIT 2023-07-29T14:21:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root 4w REG 8,1 5499631 59420 /var/log/audit/audit.log 2023-07-29T14:21:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root 6u unix 0xffff89b9c8e80cc0 0t0 16640 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root 7u unix 0xffff89b9c8e7e600 0t0 16639 type=STREAM (CONNECTED) 2023-07-29T14:21:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root 8u unix 0xffff89b9c8e7d940 0t0 16641 type=STREAM (CONNECTED) 2023-07-29T14:21:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root 9u unix 0xffff89b9c8e7dd80 0t0 16642 type=STREAM (CONNECTED) 2023-07-29T14:21:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:auditd 672 675 auditd root 10u a_inode 0,14 0 2079 [eventfd:0] 2023-07-29T14:21:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root txt REG 8,1 19624 11329 /usr/sbin/sedispatch 2023-07-29T14:21:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.795 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.795 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.795 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.795 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.795 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.795 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.795 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 144728 9145 /usr/lib64/libauparse.so.0.0.0 2023-07-29T14:21:36.796 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:36.796 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.796 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.796 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root 0u unix 0xffff89b9c8e78000 0t0 16638 type=STREAM (CONNECTED) 2023-07-29T14:21:36.796 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.796 INFO:teuthology.orchestra.run.smithi045.stdout:sedispatc 674 root 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.796 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.796 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.797 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus txt REG 8,1 131896 10627 /usr/bin/dbus-broker-launch 2023-07-29T14:21:36.797 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus DEL REG 0,1 2 /memfd:dbus-broker-log 2023-07-29T14:21:36.797 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.797 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.797 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.797 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.797 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.798 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.798 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.798 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.798 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.798 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.798 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.798 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.798 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.799 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.799 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.799 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.799 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.799 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 1u unix 0xffff89b9c6be1540 0t0 22796 type=STREAM (CONNECTED) 2023-07-29T14:21:36.799 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 2u unix 0xffff89b9c6be1540 0t0 22796 type=STREAM (CONNECTED) 2023-07-29T14:21:36.799 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 3u unix 0xffff89b9c82f1980 0t0 20667 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2023-07-29T14:21:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 4u unix 0xffff89b9c8cd8880 0t0 22806 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 5u a_inode 0,14 0 2079 [eventpoll:6,7,8,9,10,11] 2023-07-29T14:21:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 6u a_inode 0,14 0 2079 [signalfd] 2023-07-29T14:21:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 7r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 8u unix 0xffff89b9c6be5500 0t0 22808 type=STREAM (CONNECTED) 2023-07-29T14:21:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 9u unix 0xffff89b9c82f1dc0 0t0 20699 type=STREAM (CONNECTED) 2023-07-29T14:21:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 10u a_inode 0,14 0 2079 [pidfd:712] 2023-07-29T14:21:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 11u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 711 dbus 12u REG 0,1 2097152 2 /memfd:dbus-broker-log (deleted) 2023-07-29T14:21:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus txt REG 8,1 236336 10626 /usr/bin/dbus-broker 2023-07-29T14:21:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus mem REG 0,19 0 19 /sys/fs/selinux/status 2023-07-29T14:21:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 1u unix 0xffff89b9c6be1540 0t0 22796 type=STREAM (CONNECTED) 2023-07-29T14:21:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 2u unix 0xffff89b9c6be1540 0t0 22796 type=STREAM (CONNECTED) 2023-07-29T14:21:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 3u sock 0,8 0t0 19620 protocol: NETLINK 2023-07-29T14:21:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 4u unix 0xffff89b9c8cd8880 0t0 22806 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 5u a_inode 0,14 0 2079 [eventpoll:6,7,8,9,10,11,12,13,15,16,18] 2023-07-29T14:21:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 6u a_inode 0,14 0 2079 [signalfd] 2023-07-29T14:21:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 7u unix 0xffff89b9c82f1980 0t0 20667 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2023-07-29T14:21:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 8u unix 0xffff89b9c82f3fc0 0t0 1693 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-07-29T14:21:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 9u unix 0xffff89b9c6be4c80 0t0 22809 type=STREAM (CONNECTED) 2023-07-29T14:21:36.804 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 10u unix 0xffff89b9c82f2a80 0t0 1695 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-07-29T14:21:36.804 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 11u unix 0xffff89b9c82f3b80 0t0 1699 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-07-29T14:21:36.804 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 12u unix 0xffff89b9c7b472c0 0t0 2452 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-07-29T14:21:36.804 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 13u unix 0xffff89b9c7b43740 0t0 2454 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-07-29T14:21:36.804 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 15u unix 0xffff89b9c8e7a200 0t0 2960 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-07-29T14:21:36.804 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 16u unix 0xffff89b9c449a640 0t0 16699 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-07-29T14:21:36.804 INFO:teuthology.orchestra.run.smithi045.stdout:dbus-brok 712 dbus 18u unix 0xffff89b9c6be4400 0t0 2531 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-07-29T14:21:36.804 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.805 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.805 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root txt REG 8,1 3537496 11048 /usr/sbin/NetworkManager 2023-07-29T14:21:36.805 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 317560 45279 /usr/lib64/NetworkManager/1.41.8-1.el9/libnm-settings-plugin-ifcfg-rh.so 2023-07-29T14:21:36.805 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:36.805 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:36.805 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 62456 9196 /usr/lib64/libjansson.so.4.14.0 2023-07-29T14:21:36.805 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 31792 10368 /usr/lib64/libteamdctl.so.0.1.5 2023-07-29T14:21:36.806 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 52920 46945 /usr/lib64/NetworkManager/1.41.8-1.el9/libnm-device-plugin-team.so 2023-07-29T14:21:36.806 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.806 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:36.806 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.806 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:36.806 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.806 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:36.806 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:36.807 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:36.807 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.807 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.807 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.807 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.807 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.807 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.807 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.808 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.808 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:36.808 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:36.808 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.808 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:36.808 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:36.808 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.809 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.809 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.809 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.809 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:36.809 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:36.809 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:36.809 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:36.809 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.810 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.810 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.810 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.810 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.810 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 626152 9403 /usr/lib64/libhogweed.so.6.5 2023-07-29T14:21:36.810 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 340456 9405 /usr/lib64/libnettle.so.8.5 2023-07-29T14:21:36.810 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 94112 9315 /usr/lib64/libtasn1.so.6.6.0 2023-07-29T14:21:36.810 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:36.811 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:36.811 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.811 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.811 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:36.811 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.811 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.811 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.811 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:36.812 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 2291592 9409 /usr/lib64/libgnutls.so.30.33.1 2023-07-29T14:21:36.812 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:36.812 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.812 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.812 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.812 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.812 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.812 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 32408 10748 /usr/lib64/libndp.so.0.2.0 2023-07-29T14:21:36.813 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.813 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:36.813 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:36.813 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:36.813 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:36.813 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:36.813 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:36.814 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:36.814 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.814 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:36.814 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.814 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 1u unix 0xffff89b9d821bfc0 0t0 1669 type=STREAM (CONNECTED) 2023-07-29T14:21:36.814 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 2u unix 0xffff89b9d821bfc0 0t0 1669 type=STREAM (CONNECTED) 2023-07-29T14:21:36.814 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 3u a_inode 0,14 0 2079 [eventfd:3] 2023-07-29T14:21:36.814 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 4u a_inode 0,14 0 2079 [eventfd:4] 2023-07-29T14:21:36.815 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 5u unix 0xffff89b9c39b6a40 0t0 19666 type=DGRAM (UNCONNECTED) 2023-07-29T14:21:36.815 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 6u unix 0xffff89b9c7b41dc0 0t0 19667 type=STREAM (CONNECTED) 2023-07-29T14:21:36.815 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 7u a_inode 0,14 0 2079 [eventfd:5] 2023-07-29T14:21:36.815 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 8r REG 0,4 0 4026531840 net 2023-07-29T14:21:36.815 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 9r REG 0,4 0 4026532495 mnt 2023-07-29T14:21:36.815 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 10u netlink 0t0 19671 KOBJECT_UEVENT 2023-07-29T14:21:36.815 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 11u netlink 0t0 19672 GENERIC 2023-07-29T14:21:36.815 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 12u netlink 0t0 19673 GENERIC 2023-07-29T14:21:36.816 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 13u netlink 0t0 19674 ROUTE 2023-07-29T14:21:36.816 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 14r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.816 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 15r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.816 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 16u netlink 0t0 1750 KOBJECT_UEVENT 2023-07-29T14:21:36.816 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 17u a_inode 0,14 0 2079 [eventpoll] 2023-07-29T14:21:36.816 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 18u unix 0xffff89b9c449bfc0 0t0 2517 type=STREAM (CONNECTED) 2023-07-29T14:21:36.816 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 19u a_inode 0,14 0 2079 [eventpoll:21,22,26] 2023-07-29T14:21:36.816 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 20w FIFO 0,24 0t0 1313 /run/systemd/inhibit/1.ref 2023-07-29T14:21:36.817 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 21u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.817 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 22u pack 1829 0t0 IP type=SOCK_DGRAM 2023-07-29T14:21:36.817 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 23u a_inode 0,14 0 2079 [eventpoll:24,25] 2023-07-29T14:21:36.817 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 24u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.817 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 25u pack 1850 0t0 ARP type=SOCK_DGRAM 2023-07-29T14:21:36.817 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 root 26u IPv4 1851 0t0 UDP smithi045.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2023-07-29T14:21:36.817 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.818 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.818 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root txt REG 8,1 3537496 11048 /usr/sbin/NetworkManager 2023-07-29T14:21:36.818 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 317560 45279 /usr/lib64/NetworkManager/1.41.8-1.el9/libnm-settings-plugin-ifcfg-rh.so 2023-07-29T14:21:36.818 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:36.818 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:36.818 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 62456 9196 /usr/lib64/libjansson.so.4.14.0 2023-07-29T14:21:36.818 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 31792 10368 /usr/lib64/libteamdctl.so.0.1.5 2023-07-29T14:21:36.818 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 52920 46945 /usr/lib64/NetworkManager/1.41.8-1.el9/libnm-device-plugin-team.so 2023-07-29T14:21:36.819 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.819 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:36.819 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.819 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:36.819 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.819 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:36.819 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:36.819 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:36.820 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.820 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.820 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.820 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.820 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.820 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.820 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.820 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 626152 9403 /usr/lib64/libhogweed.so.6.5 2023-07-29T14:21:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 340456 9405 /usr/lib64/libnettle.so.8.5 2023-07-29T14:21:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 94112 9315 /usr/lib64/libtasn1.so.6.6.0 2023-07-29T14:21:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 2291592 9409 /usr/lib64/libgnutls.so.30.33.1 2023-07-29T14:21:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 32408 10748 /usr/lib64/libndp.so.0.2.0 2023-07-29T14:21:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.826 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:36.826 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:36.826 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:36.826 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:36.826 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:36.826 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:36.826 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:36.826 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.827 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:36.827 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.827 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 1u unix 0xffff89b9d821bfc0 0t0 1669 type=STREAM (CONNECTED) 2023-07-29T14:21:36.827 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 2u unix 0xffff89b9d821bfc0 0t0 1669 type=STREAM (CONNECTED) 2023-07-29T14:21:36.827 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 3u a_inode 0,14 0 2079 [eventfd:3] 2023-07-29T14:21:36.827 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 4u a_inode 0,14 0 2079 [eventfd:4] 2023-07-29T14:21:36.827 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 5u unix 0xffff89b9c39b6a40 0t0 19666 type=DGRAM (UNCONNECTED) 2023-07-29T14:21:36.827 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 6u unix 0xffff89b9c7b41dc0 0t0 19667 type=STREAM (CONNECTED) 2023-07-29T14:21:36.828 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 7u a_inode 0,14 0 2079 [eventfd:5] 2023-07-29T14:21:36.828 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 8r REG 0,4 0 4026531840 net 2023-07-29T14:21:36.828 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 9r REG 0,4 0 4026532495 mnt 2023-07-29T14:21:36.828 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 10u netlink 0t0 19671 KOBJECT_UEVENT 2023-07-29T14:21:36.828 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 11u netlink 0t0 19672 GENERIC 2023-07-29T14:21:36.828 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 12u netlink 0t0 19673 GENERIC 2023-07-29T14:21:36.828 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 13u netlink 0t0 19674 ROUTE 2023-07-29T14:21:36.828 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 14r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.829 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 15r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.829 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 16u netlink 0t0 1750 KOBJECT_UEVENT 2023-07-29T14:21:36.829 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 17u a_inode 0,14 0 2079 [eventpoll] 2023-07-29T14:21:36.829 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 18u unix 0xffff89b9c449bfc0 0t0 2517 type=STREAM (CONNECTED) 2023-07-29T14:21:36.829 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 19u a_inode 0,14 0 2079 [eventpoll:21,22,26] 2023-07-29T14:21:36.829 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 20w FIFO 0,24 0t0 1313 /run/systemd/inhibit/1.ref 2023-07-29T14:21:36.829 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 21u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.830 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 22u pack 1829 0t0 IP type=SOCK_DGRAM 2023-07-29T14:21:36.830 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 23u a_inode 0,14 0 2079 [eventpoll:24,25] 2023-07-29T14:21:36.830 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 24u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.830 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 25u pack 1850 0t0 ARP type=SOCK_DGRAM 2023-07-29T14:21:36.830 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 735 gmain root 26u IPv4 1851 0t0 UDP smithi045.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2023-07-29T14:21:36.830 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.830 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.830 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root txt REG 8,1 3537496 11048 /usr/sbin/NetworkManager 2023-07-29T14:21:36.831 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 317560 45279 /usr/lib64/NetworkManager/1.41.8-1.el9/libnm-settings-plugin-ifcfg-rh.so 2023-07-29T14:21:36.831 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:36.831 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:36.831 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 62456 9196 /usr/lib64/libjansson.so.4.14.0 2023-07-29T14:21:36.831 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 31792 10368 /usr/lib64/libteamdctl.so.0.1.5 2023-07-29T14:21:36.831 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 52920 46945 /usr/lib64/NetworkManager/1.41.8-1.el9/libnm-device-plugin-team.so 2023-07-29T14:21:36.831 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.831 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:36.832 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.832 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:36.832 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.832 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:36.832 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:36.832 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:36.832 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.833 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.833 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.833 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.833 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.833 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.833 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.833 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.833 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:36.834 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:36.834 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.834 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:36.834 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:36.834 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.834 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.834 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.834 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.835 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:36.835 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:36.835 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:36.835 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:36.835 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.835 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.835 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.835 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.836 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.836 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 626152 9403 /usr/lib64/libhogweed.so.6.5 2023-07-29T14:21:36.836 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 340456 9405 /usr/lib64/libnettle.so.8.5 2023-07-29T14:21:36.836 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 94112 9315 /usr/lib64/libtasn1.so.6.6.0 2023-07-29T14:21:36.836 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:36.836 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:36.836 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.836 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.837 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:36.837 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.837 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.837 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.837 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:36.837 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 2291592 9409 /usr/lib64/libgnutls.so.30.33.1 2023-07-29T14:21:36.837 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:36.838 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.838 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.838 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.838 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.838 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.838 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 32408 10748 /usr/lib64/libndp.so.0.2.0 2023-07-29T14:21:36.838 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.840 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:36.840 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.840 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 1u unix 0xffff89b9d821bfc0 0t0 1669 type=STREAM (CONNECTED) 2023-07-29T14:21:36.840 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 2u unix 0xffff89b9d821bfc0 0t0 1669 type=STREAM (CONNECTED) 2023-07-29T14:21:36.840 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 3u a_inode 0,14 0 2079 [eventfd:3] 2023-07-29T14:21:36.840 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 4u a_inode 0,14 0 2079 [eventfd:4] 2023-07-29T14:21:36.840 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 5u unix 0xffff89b9c39b6a40 0t0 19666 type=DGRAM (UNCONNECTED) 2023-07-29T14:21:36.840 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 6u unix 0xffff89b9c7b41dc0 0t0 19667 type=STREAM (CONNECTED) 2023-07-29T14:21:36.841 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 7u a_inode 0,14 0 2079 [eventfd:5] 2023-07-29T14:21:36.841 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 8r REG 0,4 0 4026531840 net 2023-07-29T14:21:36.841 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 9r REG 0,4 0 4026532495 mnt 2023-07-29T14:21:36.841 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 10u netlink 0t0 19671 KOBJECT_UEVENT 2023-07-29T14:21:36.841 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 11u netlink 0t0 19672 GENERIC 2023-07-29T14:21:36.841 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 12u netlink 0t0 19673 GENERIC 2023-07-29T14:21:36.841 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 13u netlink 0t0 19674 ROUTE 2023-07-29T14:21:36.841 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 14r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.842 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 15r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.842 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 16u netlink 0t0 1750 KOBJECT_UEVENT 2023-07-29T14:21:36.842 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 17u a_inode 0,14 0 2079 [eventpoll] 2023-07-29T14:21:36.842 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 18u unix 0xffff89b9c449bfc0 0t0 2517 type=STREAM (CONNECTED) 2023-07-29T14:21:36.842 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 19u a_inode 0,14 0 2079 [eventpoll:21,22,26] 2023-07-29T14:21:36.842 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 20w FIFO 0,24 0t0 1313 /run/systemd/inhibit/1.ref 2023-07-29T14:21:36.842 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 21u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.843 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 22u pack 1829 0t0 IP type=SOCK_DGRAM 2023-07-29T14:21:36.843 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 23u a_inode 0,14 0 2079 [eventpoll:24,25] 2023-07-29T14:21:36.843 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 24u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.843 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 25u pack 1850 0t0 ARP type=SOCK_DGRAM 2023-07-29T14:21:36.843 INFO:teuthology.orchestra.run.smithi045.stdout:NetworkMa 713 736 gdbus root 26u IPv4 1851 0t0 UDP smithi045.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2023-07-29T14:21:36.843 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.843 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.843 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root txt REG 8,1 61904 12068 /usr/sbin/irqbalance 2023-07-29T14:21:36.844 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.844 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.844 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.844 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:36.844 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.844 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.844 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.844 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.845 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root 1u unix 0xffff89b9d821ee80 0t0 1678 type=STREAM (CONNECTED) 2023-07-29T14:21:36.845 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root 2u unix 0xffff89b9d821ee80 0t0 1678 type=STREAM (CONNECTED) 2023-07-29T14:21:36.845 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root 3u a_inode 0,14 0 2079 [eventfd:1] 2023-07-29T14:21:36.845 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root 4u a_inode 0,14 0 2079 [eventfd:2] 2023-07-29T14:21:36.845 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 root 5u unix 0xffff89b9d821d500 0t0 1702 /run/irqbalance/irqbalance716.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.845 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.845 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.846 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root txt REG 8,1 61904 12068 /usr/sbin/irqbalance 2023-07-29T14:21:36.846 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.846 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.846 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.846 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:36.846 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.846 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.846 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.847 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.847 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root 1u unix 0xffff89b9d821ee80 0t0 1678 type=STREAM (CONNECTED) 2023-07-29T14:21:36.847 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root 2u unix 0xffff89b9d821ee80 0t0 1678 type=STREAM (CONNECTED) 2023-07-29T14:21:36.847 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root 3u a_inode 0,14 0 2079 [eventfd:1] 2023-07-29T14:21:36.847 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root 4u a_inode 0,14 0 2079 [eventfd:2] 2023-07-29T14:21:36.847 INFO:teuthology.orchestra.run.smithi045.stdout:irqbalanc 716 724 gmain root 5u unix 0xffff89b9d821d500 0t0 1702 /run/irqbalance/irqbalance716.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.847 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.847 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.848 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt txt REG 8,1 32280 10707 /usr/bin/lsmd 2023-07-29T14:21:36.848 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.848 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt mem REG 8,1 56912 10225 /usr/lib64/libconfig.so.11.0.2 2023-07-29T14:21:36.848 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.848 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.848 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt 1u unix 0xffff89b9c7b42640 0t0 19613 type=STREAM (CONNECTED) 2023-07-29T14:21:36.848 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt 2u unix 0xffff89b9c7b42640 0t0 19613 type=STREAM (CONNECTED) 2023-07-29T14:21:36.848 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt 3uW REG 0,24 0 1265 /run/lsm/ipc/.lsmd-ipc-lock 2023-07-29T14:21:36.849 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt 5u unix 0xffff89b9d81661c0 0t0 17990 /var/run/lsm/ipc/simc type=STREAM (LISTEN) 2023-07-29T14:21:36.849 INFO:teuthology.orchestra.run.smithi045.stdout:lsmd 717 libstoragemgmt 6u unix 0xffff89b9d8161540 0t0 17991 /var/run/lsm/ipc/sim type=STREAM (LISTEN) 2023-07-29T14:21:36.849 INFO:teuthology.orchestra.run.smithi045.stdout:mcelog 718 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.849 INFO:teuthology.orchestra.run.smithi045.stdout:mcelog 718 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.849 INFO:teuthology.orchestra.run.smithi045.stdout:mcelog 718 root txt REG 8,1 180192 11906 /usr/sbin/mcelog 2023-07-29T14:21:36.849 INFO:teuthology.orchestra.run.smithi045.stdout:mcelog 718 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.849 INFO:teuthology.orchestra.run.smithi045.stdout:mcelog 718 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.849 INFO:teuthology.orchestra.run.smithi045.stdout:mcelog 718 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.850 INFO:teuthology.orchestra.run.smithi045.stdout:mcelog 718 root 1u unix 0xffff89b9c7b41540 0t0 19616 type=STREAM (CONNECTED) 2023-07-29T14:21:36.850 INFO:teuthology.orchestra.run.smithi045.stdout:mcelog 718 root 2u unix 0xffff89b9c7b41540 0t0 19616 type=STREAM (CONNECTED) 2023-07-29T14:21:36.850 INFO:teuthology.orchestra.run.smithi045.stdout:mcelog 718 root 3r CHR 10,227 0t0 142 /dev/mcelog 2023-07-29T14:21:36.850 INFO:teuthology.orchestra.run.smithi045.stdout:mcelog 718 root 4u unix 0xffff89b9c7b45500 0t0 19631 /var/run/mcelog-client type=STREAM (LISTEN) 2023-07-29T14:21:36.850 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.850 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.850 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.851 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.851 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.851 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.851 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.851 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.851 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.851 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.851 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.852 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.852 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.852 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.852 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.852 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.852 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.852 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.852 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.853 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.853 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.853 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.853 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.853 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.853 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.853 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.853 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.854 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.854 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.854 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.854 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.854 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.854 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.854 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.854 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.855 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.855 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.855 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.855 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.855 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.855 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.855 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.855 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.856 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.856 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.856 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.856 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.856 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.856 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.856 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.857 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.857 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.857 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.857 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.857 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.857 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.857 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.857 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.858 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.858 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.858 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.858 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.858 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.858 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.858 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.858 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.859 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.859 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.859 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.859 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.859 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.859 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.859 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.859 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.860 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.860 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.860 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.860 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.860 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.860 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.860 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.860 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 741 gmain polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.861 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.861 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.861 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.861 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.861 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.861 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.861 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.861 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.862 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.862 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.862 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.862 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.862 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.862 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.862 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.863 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.863 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.863 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.863 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.863 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.863 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.863 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.863 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.864 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.864 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.864 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.864 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.864 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.864 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.864 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.864 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.865 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.865 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.865 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.865 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.865 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.865 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.865 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.866 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.866 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.866 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 743 gdbus polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.866 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.866 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.866 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.866 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.866 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.867 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.867 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.867 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.867 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.867 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.867 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.867 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.867 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.868 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.868 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.868 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.868 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.868 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.868 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.868 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.868 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.869 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.869 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.869 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.869 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.869 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.869 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.869 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.869 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.870 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.870 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.870 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.870 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.870 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.870 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.870 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.870 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 744 JS\x20Hel polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.873 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.873 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.873 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.873 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.873 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.873 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.873 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.873 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.874 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.874 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.874 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.874 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.874 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.874 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.874 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.874 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.875 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.875 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.875 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.875 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.875 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.875 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.875 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.875 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.876 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.876 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.876 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.876 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.876 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.876 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 745 JS\x20Hel polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.876 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.877 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.877 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.877 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.877 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.877 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.877 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.877 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.877 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.878 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.878 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.878 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.878 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.878 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.878 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.878 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.878 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.879 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.879 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.879 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.879 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.879 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.879 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.879 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.880 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.880 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.880 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.880 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.880 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.880 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.880 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.880 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.881 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.881 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.881 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.881 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.881 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.881 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.881 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.881 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 746 JS\x20Hel polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.882 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.883 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.883 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.883 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.883 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.883 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.883 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.883 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.883 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.884 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.884 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.884 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.884 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.884 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.884 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.884 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.884 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.885 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.885 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.885 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.885 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.885 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.885 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.885 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.885 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.886 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.886 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.886 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.886 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.886 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.886 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.886 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.887 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.887 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.887 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 747 JS\x20Hel polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.887 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.887 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.887 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.887 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.887 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.888 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.888 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.888 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.888 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.888 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.888 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.888 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.888 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.889 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.889 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.889 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.889 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.889 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.889 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.889 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.889 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.890 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.890 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.890 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.890 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.890 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.890 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.890 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.890 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.891 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.891 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.891 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.891 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.891 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.891 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.891 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.892 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.892 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.892 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.892 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.892 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 748 JS\x20Hel polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.892 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.892 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.892 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.893 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.893 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.893 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.893 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.893 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.893 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.893 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.893 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.894 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.894 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.894 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.894 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.894 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.894 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.894 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.894 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.895 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.895 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.895 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.895 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.895 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.895 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.895 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.896 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.896 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.896 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.896 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.896 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.896 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.896 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.896 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.897 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.897 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.897 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.897 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.897 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.897 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.897 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 749 JS\x20Hel polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.897 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.898 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.898 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.898 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.898 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.898 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.898 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.898 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.898 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.899 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.899 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.899 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.899 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.899 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.899 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.899 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.899 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.900 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.900 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.900 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.900 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.900 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.900 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.900 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.901 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.901 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.901 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.901 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.901 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.901 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.901 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.901 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.902 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.902 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.902 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.902 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.902 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.902 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.902 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.902 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 750 JS\x20Hel polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.904 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.904 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.904 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.904 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.904 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.904 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.904 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.905 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.905 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.905 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.905 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.905 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.905 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.905 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.905 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.906 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.906 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.906 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.906 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.906 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.906 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.906 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.906 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.907 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.907 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.907 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.907 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.907 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.907 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.907 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.907 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.908 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.908 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.908 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 751 JS\x20Hel polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.908 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.908 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.908 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd txt REG 8,1 146624 43518 /usr/lib/polkit-1/polkitd 2023-07-29T14:21:36.908 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.908 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.909 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.909 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.909 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.909 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.909 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.909 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:36.909 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.909 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.910 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.910 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 29047400 31108 /usr/lib64/polkit-1/libmozjs-78.so 2023-07-29T14:21:36.910 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.910 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.910 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.910 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.910 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.910 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.911 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.911 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.911 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.911 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.911 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.912 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.912 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.912 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 132488 9593 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-07-29T14:21:36.912 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.912 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.912 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.912 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.912 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd 3r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.913 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd 4u a_inode 0,14 0 2079 [eventfd:6] 2023-07-29T14:21:36.913 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd 5u a_inode 0,14 0 2079 [eventfd:7] 2023-07-29T14:21:36.913 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd 6u unix 0xffff89b9c7b43b80 0t0 19681 type=STREAM (CONNECTED) 2023-07-29T14:21:36.913 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd 7u a_inode 0,14 0 2079 [eventfd:8] 2023-07-29T14:21:36.913 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd 8r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.913 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd 9u a_inode 0,14 0 2079 [eventfd:13] 2023-07-29T14:21:36.913 INFO:teuthology.orchestra.run.smithi045.stdout:polkitd 719 765 polkitd polkitd 10u unix 0xffff89b9c8cdd0c0 0t0 22839 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.913 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.914 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.914 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root txt REG 8,1 306888 16849 /usr/lib/systemd/systemd-logind 2023-07-29T14:21:36.914 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 594568 1228 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-07-29T14:21:36.914 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 357408 9535 /usr/lib64/libnss_systemd.so.2 2023-07-29T14:21:36.914 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.914 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.914 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.915 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.915 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.915 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.915 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.915 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.915 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.915 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.915 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.916 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:36.916 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.916 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 28552 9218 /usr/lib64/libattr.so.1.1.2501 2023-07-29T14:21:36.916 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 127680 9386 /usr/lib64/libseccomp.so.2.5.2 2023-07-29T14:21:36.916 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:36.916 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.916 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.916 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 107984 9595 /usr/lib64/libkmod.so.2.3.6 2023-07-29T14:21:36.917 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 3640680 43416 /usr/lib64/systemd/libsystemd-shared-252.so 2023-07-29T14:21:36.917 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:36.917 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.917 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:36.917 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.917 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.917 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 41120 9220 /usr/lib64/libacl.so.1.1.2301 2023-07-29T14:21:36.917 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.918 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 21219 11939 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-07-29T14:21:36.918 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.918 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-07-29T14:21:36.918 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.918 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 1u unix 0xffff89b9c82f0440 0t0 20700 type=STREAM (CONNECTED) 2023-07-29T14:21:36.918 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 2u unix 0xffff89b9c82f0440 0t0 20700 type=STREAM (CONNECTED) 2023-07-29T14:21:36.918 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 3u unix 0xffff89b9c8311980 0t0 20725 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.918 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 4u a_inode 0,14 0 2079 [eventpoll:5,6,7,8,9,10,11,12,13,14,15,16,17,18,20] 2023-07-29T14:21:36.919 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 5u a_inode 0,14 0 2079 [signalfd] 2023-07-29T14:21:36.919 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 6u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.919 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 7r REG 0,21 4096 22170 /sys/devices/virtual/tty/tty0/active 2023-07-29T14:21:36.919 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 8u netlink 0t0 20736 KOBJECT_UEVENT 2023-07-29T14:21:36.919 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 9r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.919 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 10r FIFO 0,24 0t0 1313 /run/systemd/inhibit/1.ref 2023-07-29T14:21:36.919 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 11u netlink 0t0 20737 KOBJECT_UEVENT 2023-07-29T14:21:36.920 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 12u netlink 0t0 20738 KOBJECT_UEVENT 2023-07-29T14:21:36.920 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 13u netlink 0t0 20739 KOBJECT_UEVENT 2023-07-29T14:21:36.920 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 14u unix 0xffff89b9c82f1100 0t0 20740 type=STREAM (CONNECTED) 2023-07-29T14:21:36.920 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 15u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:36.920 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 16u CHR 13,65 0t0 138 /dev/input/event1 2023-07-29T14:21:36.920 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 17u CHR 13,64 0t0 137 /dev/input/event0 2023-07-29T14:21:36.920 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 18u CHR 13,66 0t0 148 /dev/input/event2 2023-07-29T14:21:36.920 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 19u CHR 4,6 0t0 25 /dev/tty6 2023-07-29T14:21:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:systemd-l 723 root 20r FIFO 0,24 0t0 1768 /run/systemd/sessions/4.ref 2023-07-29T14:21:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root txt REG 8,1 956448 11769 /usr/sbin/sshd 2023-07-29T14:21:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.922 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.922 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.922 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.922 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.922 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.922 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.922 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.922 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.923 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.923 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.923 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.923 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:36.923 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.923 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.923 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.923 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.924 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.924 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:36.924 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.924 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.924 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:36.924 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.924 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.924 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.925 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root 1u unix 0xffff89b9c6be2200 0t0 22836 type=STREAM (CONNECTED) 2023-07-29T14:21:36.925 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root 2u unix 0xffff89b9c6be2200 0t0 22836 type=STREAM (CONNECTED) 2023-07-29T14:21:36.925 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root 3u IPv4 19738 0t0 TCP *:ssh (LISTEN) 2023-07-29T14:21:36.925 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 740 root 4u IPv6 19740 0t0 TCP *:ssh (LISTEN) 2023-07-29T14:21:36.925 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.925 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.925 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root txt REG 8,1 15448 9913 /usr/bin/python3.9 2023-07-29T14:21:36.925 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:36.926 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 130456 32336 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.926 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 107744 32339 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.926 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 204696 31136 /usr/lib64/girepository-1.0/GLib-2.0.typelib 2023-07-29T14:21:36.926 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.926 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.926 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.926 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.927 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.927 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.927 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.927 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.927 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 143200 9609 /usr/lib64/libgirepository-1.0.so.1.0.0 2023-07-29T14:21:36.927 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 357112 33457 /usr/lib64/python3.9/site-packages/gi/_gi.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.927 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.927 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.928 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 24736 33337 /usr/lib64/python3.9/site-packages/_dbus_glib_bindings.so 2023-07-29T14:21:36.928 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.928 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.928 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.928 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.928 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.928 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.928 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:36.929 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.929 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 71416 32375 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.929 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.929 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.929 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 171376 33336 /usr/lib64/python3.9/site-packages/_dbus_bindings.so 2023-07-29T14:21:36.929 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 23800 32354 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.929 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.929 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 46560 32348 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.930 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 76792 9115 /usr/lib64/libbz2.so.1.0.8 2023-07-29T14:21:36.930 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 29232 32326 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.930 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 41752 32384 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.930 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 50224 32325 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.930 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.930 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 20000 32370 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.930 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 69040 32344 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.931 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 24160 32356 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.931 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 15984 32324 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.931 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 15560 32351 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.931 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 124192 32352 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.931 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 56416 32361 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.931 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 68000 32365 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.931 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 23832 32369 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.931 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 37592 32378 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.932 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 101216 32357 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.932 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.932 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 71616 32371 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.932 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 28064 32345 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.932 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.932 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 3474032 9915 /usr/lib64/libpython3.9.so.1.0 2023-07-29T14:21:36.932 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.932 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.933 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 20528 32355 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.933 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.933 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.933 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root 1u unix 0xffff89b9c7b42ec0 0t0 19678 type=STREAM (CONNECTED) 2023-07-29T14:21:36.933 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root 2u unix 0xffff89b9c7b42ec0 0t0 19678 type=STREAM (CONNECTED) 2023-07-29T14:21:36.933 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root 3w REG 8,1 15044 1713 /var/log/tuned/tuned.log 2023-07-29T14:21:36.933 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root 4u sock 0,8 0t0 20831 protocol: NETLINK 2023-07-29T14:21:36.933 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root 5u unix 0xffff89b9c6be3b80 0t0 22894 type=STREAM (CONNECTED) 2023-07-29T14:21:36.934 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root 6u a_inode 0,14 0 2079 [eventfd:17] 2023-07-29T14:21:36.934 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root 7u unix 0xffff89b9c8e7aec0 0t0 16745 type=STREAM (CONNECTED) 2023-07-29T14:21:36.934 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 root 8u unix 0xffff89b9c8e7f2c0 0t0 16746 type=STREAM (CONNECTED) 2023-07-29T14:21:36.934 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.934 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.934 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root txt REG 8,1 15448 9913 /usr/bin/python3.9 2023-07-29T14:21:36.934 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:36.934 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 130456 32336 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 107744 32339 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 204696 31136 /usr/lib64/girepository-1.0/GLib-2.0.typelib 2023-07-29T14:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 1932640 9580 /usr/lib64/libgio-2.0.so.0.6800.4 2023-07-29T14:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 19480 9584 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-07-29T14:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 375136 9586 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-07-29T14:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 143200 9609 /usr/lib64/libgirepository-1.0.so.1.0.0 2023-07-29T14:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 357112 33457 /usr/lib64/python3.9/site-packages/gi/_gi.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 489696 9414 /usr/lib64/libpcre.so.1.2.12 2023-07-29T14:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 1298936 9582 /usr/lib64/libglib-2.0.so.0.6800.4 2023-07-29T14:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 24736 33337 /usr/lib64/python3.9/site-packages/_dbus_glib_bindings.so 2023-07-29T14:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 197952 9190 /usr/lib64/libexpat.so.1.8.10 2023-07-29T14:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 71416 32375 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 171376 33336 /usr/lib64/python3.9/site-packages/_dbus_bindings.so 2023-07-29T14:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 23800 32354 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 46560 32348 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 76792 9115 /usr/lib64/libbz2.so.1.0.8 2023-07-29T14:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 29232 32326 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 41752 32384 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 50224 32325 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 20000 32370 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 69040 32344 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 24160 32356 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 15984 32324 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 15560 32351 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 124192 32352 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 56416 32361 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 68000 32365 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 23832 32369 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 37592 32378 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 101216 32357 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 71616 32371 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 28064 32345 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 3474032 9915 /usr/lib64/libpython3.9.so.1.0 2023-07-29T14:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 20528 32355 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root 1u unix 0xffff89b9c7b42ec0 0t0 19678 type=STREAM (CONNECTED) 2023-07-29T14:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root 2u unix 0xffff89b9c7b42ec0 0t0 19678 type=STREAM (CONNECTED) 2023-07-29T14:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root 3w REG 8,1 15044 1713 /var/log/tuned/tuned.log 2023-07-29T14:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root 4u sock 0,8 0t0 20831 protocol: NETLINK 2023-07-29T14:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root 5u unix 0xffff89b9c6be3b80 0t0 22894 type=STREAM (CONNECTED) 2023-07-29T14:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root 6u a_inode 0,14 0 2079 [eventfd:17] 2023-07-29T14:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root 7u unix 0xffff89b9c8e7aec0 0t0 16745 type=STREAM (CONNECTED) 2023-07-29T14:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout:tuned 742 860 tuned root 8u unix 0xffff89b9c8e7f2c0 0t0 16746 type=STREAM (CONNECTED) 2023-07-29T14:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root txt REG 8,1 128840 2325 /usr/sbin/gssproxy 2023-07-29T14:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 70960 2316 /usr/lib64/libev.so.4.0.0 2023-07-29T14:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 16168 2319 /usr/lib64/libverto-libev.so.1.0.0 2023-07-29T14:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 15912 9178 /usr/lib64/libbasicobjects.so.0.1.0 2023-07-29T14:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 20304 9382 /usr/lib64/libpath_utils.so.1.0.1 2023-07-29T14:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 58392 9180 /usr/lib64/libcollection.so.4.1.1 2023-07-29T14:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 143040 10810 /usr/lib64/libgssrpc.so.4.2 2023-07-29T14:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 16056 9182 /usr/lib64/libref_array.so.1.2.1 2023-07-29T14:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 120640 9384 /usr/lib64/libini_config.so.5.2.1 2023-07-29T14:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 28472 10806 /usr/lib64/libverto.so.1.0.0 2023-07-29T14:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 58368 9103 /usr/lib64/libpopt.so.0.0.1 2023-07-29T14:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root 1u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root 2u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root 3u unix 0xffff89b9c39b2a80 0t0 19695 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root 4r FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root 5u a_inode 0,14 0 2079 [eventpoll:4,6,7,8] 2023-07-29T14:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root 6u a_inode 0,14 0 2079 [eventfd:12] 2023-07-29T14:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root 7u unix 0xffff89b9c7b450c0 0t0 19696 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root 8u unix 0xffff89b9c7b40cc0 0t0 19697 /run/gssproxy.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 root 9w FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root txt REG 8,1 128840 2325 /usr/sbin/gssproxy 2023-07-29T14:21:36.950 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:36.950 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.950 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.950 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 70960 2316 /usr/lib64/libev.so.4.0.0 2023-07-29T14:21:36.950 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 16168 2319 /usr/lib64/libverto-libev.so.1.0.0 2023-07-29T14:21:36.950 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.950 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 15912 9178 /usr/lib64/libbasicobjects.so.0.1.0 2023-07-29T14:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 20304 9382 /usr/lib64/libpath_utils.so.1.0.1 2023-07-29T14:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 58392 9180 /usr/lib64/libcollection.so.4.1.1 2023-07-29T14:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 143040 10810 /usr/lib64/libgssrpc.so.4.2 2023-07-29T14:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 16056 9182 /usr/lib64/libref_array.so.1.2.1 2023-07-29T14:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 120640 9384 /usr/lib64/libini_config.so.5.2.1 2023-07-29T14:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 28472 10806 /usr/lib64/libverto.so.1.0.0 2023-07-29T14:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 58368 9103 /usr/lib64/libpopt.so.0.0.1 2023-07-29T14:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root 1u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root 2u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root 3u unix 0xffff89b9c39b2a80 0t0 19695 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root 4r FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root 5u a_inode 0,14 0 2079 [eventpoll:4,6,7,8] 2023-07-29T14:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root 6u a_inode 0,14 0 2079 [eventfd:12] 2023-07-29T14:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root 7u unix 0xffff89b9c7b450c0 0t0 19696 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root 8u unix 0xffff89b9c7b40cc0 0t0 19697 /run/gssproxy.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 758 gssproxy root 9w FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.956 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.956 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.956 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root txt REG 8,1 128840 2325 /usr/sbin/gssproxy 2023-07-29T14:21:36.956 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:36.956 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.956 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.956 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 70960 2316 /usr/lib64/libev.so.4.0.0 2023-07-29T14:21:36.956 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 16168 2319 /usr/lib64/libverto-libev.so.1.0.0 2023-07-29T14:21:36.957 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.957 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.957 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.957 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.957 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:36.957 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:36.957 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:36.958 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.958 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 15912 9178 /usr/lib64/libbasicobjects.so.0.1.0 2023-07-29T14:21:36.958 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 20304 9382 /usr/lib64/libpath_utils.so.1.0.1 2023-07-29T14:21:36.958 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 58392 9180 /usr/lib64/libcollection.so.4.1.1 2023-07-29T14:21:36.958 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.958 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.958 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.958 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.959 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.959 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 143040 10810 /usr/lib64/libgssrpc.so.4.2 2023-07-29T14:21:36.959 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.959 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 16056 9182 /usr/lib64/libref_array.so.1.2.1 2023-07-29T14:21:36.959 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 120640 9384 /usr/lib64/libini_config.so.5.2.1 2023-07-29T14:21:36.959 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 28472 10806 /usr/lib64/libverto.so.1.0.0 2023-07-29T14:21:36.959 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.959 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.960 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 58368 9103 /usr/lib64/libpopt.so.0.0.1 2023-07-29T14:21:36.960 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:36.960 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:36.960 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:36.960 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:36.960 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:36.960 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:36.960 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.961 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:36.961 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.961 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root 1u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.961 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root 2u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.961 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root 3u unix 0xffff89b9c39b2a80 0t0 19695 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.961 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root 4r FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.961 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root 5u a_inode 0,14 0 2079 [eventpoll:4,6,7,8] 2023-07-29T14:21:36.961 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root 6u a_inode 0,14 0 2079 [eventfd:12] 2023-07-29T14:21:36.962 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root 7u unix 0xffff89b9c7b450c0 0t0 19696 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.962 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root 8u unix 0xffff89b9c7b40cc0 0t0 19697 /run/gssproxy.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.962 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 759 gssproxy root 9w FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.962 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.962 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.962 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root txt REG 8,1 128840 2325 /usr/sbin/gssproxy 2023-07-29T14:21:36.962 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:36.962 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.963 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.963 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 70960 2316 /usr/lib64/libev.so.4.0.0 2023-07-29T14:21:36.963 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 16168 2319 /usr/lib64/libverto-libev.so.1.0.0 2023-07-29T14:21:36.963 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.963 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.963 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.963 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.963 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:36.964 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:36.964 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:36.964 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.964 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 15912 9178 /usr/lib64/libbasicobjects.so.0.1.0 2023-07-29T14:21:36.964 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 20304 9382 /usr/lib64/libpath_utils.so.1.0.1 2023-07-29T14:21:36.964 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 58392 9180 /usr/lib64/libcollection.so.4.1.1 2023-07-29T14:21:36.964 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.964 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.965 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.965 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.965 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.965 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 143040 10810 /usr/lib64/libgssrpc.so.4.2 2023-07-29T14:21:36.965 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.965 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 16056 9182 /usr/lib64/libref_array.so.1.2.1 2023-07-29T14:21:36.965 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 120640 9384 /usr/lib64/libini_config.so.5.2.1 2023-07-29T14:21:36.966 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 28472 10806 /usr/lib64/libverto.so.1.0.0 2023-07-29T14:21:36.966 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.966 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.966 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 58368 9103 /usr/lib64/libpopt.so.0.0.1 2023-07-29T14:21:36.966 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:36.966 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:36.966 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:36.966 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:36.967 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:36.967 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:36.967 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.967 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:36.967 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.967 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root 1u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.967 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root 2u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.967 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root 3u unix 0xffff89b9c39b2a80 0t0 19695 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.968 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root 4r FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.968 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root 5u a_inode 0,14 0 2079 [eventpoll:4,6,7,8] 2023-07-29T14:21:36.968 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root 6u a_inode 0,14 0 2079 [eventfd:12] 2023-07-29T14:21:36.968 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root 7u unix 0xffff89b9c7b450c0 0t0 19696 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.968 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root 8u unix 0xffff89b9c7b40cc0 0t0 19697 /run/gssproxy.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.968 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 760 gssproxy root 9w FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.968 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.968 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.969 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root txt REG 8,1 128840 2325 /usr/sbin/gssproxy 2023-07-29T14:21:36.969 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:36.969 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.969 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.969 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 70960 2316 /usr/lib64/libev.so.4.0.0 2023-07-29T14:21:36.969 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 16168 2319 /usr/lib64/libverto-libev.so.1.0.0 2023-07-29T14:21:36.969 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.970 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.970 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.970 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.970 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:36.970 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:36.970 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:36.970 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.970 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 15912 9178 /usr/lib64/libbasicobjects.so.0.1.0 2023-07-29T14:21:36.971 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 20304 9382 /usr/lib64/libpath_utils.so.1.0.1 2023-07-29T14:21:36.971 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 58392 9180 /usr/lib64/libcollection.so.4.1.1 2023-07-29T14:21:36.971 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.971 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.971 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.971 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.971 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.971 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 143040 10810 /usr/lib64/libgssrpc.so.4.2 2023-07-29T14:21:36.972 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.972 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 16056 9182 /usr/lib64/libref_array.so.1.2.1 2023-07-29T14:21:36.972 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 120640 9384 /usr/lib64/libini_config.so.5.2.1 2023-07-29T14:21:36.972 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 28472 10806 /usr/lib64/libverto.so.1.0.0 2023-07-29T14:21:36.972 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.972 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.972 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 58368 9103 /usr/lib64/libpopt.so.0.0.1 2023-07-29T14:21:36.972 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:36.973 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:36.973 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:36.973 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:36.973 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:36.973 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:36.973 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.973 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:36.973 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.974 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root 1u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.974 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root 2u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.974 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root 3u unix 0xffff89b9c39b2a80 0t0 19695 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.974 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root 4r FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.974 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root 5u a_inode 0,14 0 2079 [eventpoll:4,6,7,8] 2023-07-29T14:21:36.974 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root 6u a_inode 0,14 0 2079 [eventfd:12] 2023-07-29T14:21:36.974 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root 7u unix 0xffff89b9c7b450c0 0t0 19696 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.975 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root 8u unix 0xffff89b9c7b40cc0 0t0 19697 /run/gssproxy.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.975 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 761 gssproxy root 9w FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.975 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.975 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.975 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root txt REG 8,1 128840 2325 /usr/sbin/gssproxy 2023-07-29T14:21:36.975 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:36.975 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.975 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.976 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 70960 2316 /usr/lib64/libev.so.4.0.0 2023-07-29T14:21:36.976 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 16168 2319 /usr/lib64/libverto-libev.so.1.0.0 2023-07-29T14:21:36.976 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.976 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.976 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.976 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.976 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:36.976 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:36.977 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:36.977 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.977 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 15912 9178 /usr/lib64/libbasicobjects.so.0.1.0 2023-07-29T14:21:36.977 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 20304 9382 /usr/lib64/libpath_utils.so.1.0.1 2023-07-29T14:21:36.977 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 58392 9180 /usr/lib64/libcollection.so.4.1.1 2023-07-29T14:21:36.977 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.977 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.977 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.978 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.978 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.978 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 143040 10810 /usr/lib64/libgssrpc.so.4.2 2023-07-29T14:21:36.978 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.978 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 16056 9182 /usr/lib64/libref_array.so.1.2.1 2023-07-29T14:21:36.978 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 120640 9384 /usr/lib64/libini_config.so.5.2.1 2023-07-29T14:21:36.978 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 28472 10806 /usr/lib64/libverto.so.1.0.0 2023-07-29T14:21:36.978 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.979 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.979 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 58368 9103 /usr/lib64/libpopt.so.0.0.1 2023-07-29T14:21:36.979 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:36.979 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:36.979 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:36.979 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:36.979 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:36.979 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:36.980 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.980 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:36.980 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.980 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root 1u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.980 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root 2u unix 0xffff89b9d821cc80 0t0 1712 type=STREAM (CONNECTED) 2023-07-29T14:21:36.980 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root 3u unix 0xffff89b9c39b2a80 0t0 19695 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.980 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root 4r FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.980 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root 5u a_inode 0,14 0 2079 [eventpoll:4,6,7,8] 2023-07-29T14:21:36.981 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root 6u a_inode 0,14 0 2079 [eventfd:12] 2023-07-29T14:21:36.981 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root 7u unix 0xffff89b9c7b450c0 0t0 19696 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.981 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root 8u unix 0xffff89b9c7b40cc0 0t0 19697 /run/gssproxy.sock type=STREAM (LISTEN) 2023-07-29T14:21:36.981 INFO:teuthology.orchestra.run.smithi045.stdout:gssproxy 757 762 gssproxy root 9w FIFO 0,13 0t0 19700 pipe 2023-07-29T14:21:36.981 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root cwd DIR 8,1 4096 1685 /var/spool/at 2023-07-29T14:21:36.981 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.981 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root txt REG 8,1 32680 11887 /usr/sbin/atd 2023-07-29T14:21:36.982 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.982 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.982 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.982 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.982 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:36.982 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.982 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.982 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:36.983 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.983 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.983 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.983 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.983 INFO:teuthology.orchestra.run.smithi045.stdout:atd 764 root 3uW REG 0,24 4 1300 /run/atd.pid 2023-07-29T14:21:36.983 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.983 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.983 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root txt REG 8,1 74552 10648 /usr/sbin/crond 2023-07-29T14:21:36.984 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:36.984 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:36.984 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:36.984 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.984 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.984 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.984 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:36.984 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:36.985 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:36.985 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:36.985 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:36.985 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:36.985 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:36.985 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.985 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:36.985 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:36.986 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:36.986 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:36.986 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:36.986 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:36.986 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.986 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:36.986 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.986 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root 1u unix 0xffff89b9c8e7ea40 0t0 16701 type=STREAM (CONNECTED) 2023-07-29T14:21:36.987 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root 2u unix 0xffff89b9c8e7ea40 0t0 16701 type=STREAM (CONNECTED) 2023-07-29T14:21:36.987 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root 3uW REG 0,24 4 1303 /run/crond.pid 2023-07-29T14:21:36.987 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root 4u unix 0xffff89b9c8e82640 0t0 16705 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.987 INFO:teuthology.orchestra.run.smithi045.stdout:crond 766 root 5r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:36.987 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe cwd DIR 8,1 4096 2 / 2023-07-29T14:21:36.987 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.987 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe txt REG 8,1 80984 54567 /usr/sbin/nrpe 2023-07-29T14:21:36.987 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.988 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.988 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.988 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:36.988 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.988 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.988 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe 1w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.988 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe 2w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:36.989 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe 3u IPv4 18025 0t0 TCP *:nrpe (LISTEN) 2023-07-29T14:21:36.989 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe 4u unix 0xffff89b9c8e40000 0t0 18017 type=DGRAM (CONNECTED) 2023-07-29T14:21:36.989 INFO:teuthology.orchestra.run.smithi045.stdout:nrpe 881 nrpe 5u IPv6 18026 0t0 TCP *:nrpe (LISTEN) 2023-07-29T14:21:36.989 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp cwd DIR 8,1 4096 1610 /var/log/pcp/pmcd 2023-07-29T14:21:36.989 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.989 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp txt REG 8,1 124616 46968 /usr/libexec/pcp/bin/pmcd 2023-07-29T14:21:36.989 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 4449 53878 /usr/libexec/pcp/pmdas/jbd2/help.pag 2023-07-29T14:21:36.989 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 36392 53910 /usr/libexec/pcp/pmdas/mmv/pmda_mmv.so 2023-07-29T14:21:36.990 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 26392 53924 /usr/libexec/pcp/pmdas/pmcd/help.pag 2023-07-29T14:21:36.990 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 38240 53925 /usr/libexec/pcp/pmdas/pmcd/pmda_pmcd.so 2023-07-29T14:21:36.990 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.990 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.990 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.990 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.990 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.990 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.991 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.991 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.991 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.991 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.991 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:36.991 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.991 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.991 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.992 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:36.992 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.992 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.992 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.992 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:36.992 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.992 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:36.992 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.993 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.993 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 82624 9851 /usr/lib64/libavahi-client.so.3.2.9 2023-07-29T14:21:36.993 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 58464 9853 /usr/lib64/libavahi-common.so.3.5.4 2023-07-29T14:21:36.993 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:36.993 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:36.993 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 758088 12526 /usr/lib64/libpcp.so.3 2023-07-29T14:21:36.993 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 107344 12531 /usr/lib64/libpcp_pmda.so.3 2023-07-29T14:21:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 300 53877 /usr/libexec/pcp/pmdas/jbd2/help.dir 2023-07-29T14:21:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 16176 53879 /usr/libexec/pcp/pmdas/jbd2/pmda_jbd2.so 2023-07-29T14:21:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp mem REG 8,1 1380 53923 /usr/libexec/pcp/pmdas/pmcd/help.dir 2023-07-29T14:21:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 0u IPv4 19809 0t0 TCP localhost:pmcd (LISTEN) 2023-07-29T14:21:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 1w REG 8,1 2167 1618 /var/log/pcp/pmcd/pmcd.log 2023-07-29T14:21:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 2w REG 8,1 2167 1618 /var/log/pcp/pmcd/pmcd.log 2023-07-29T14:21:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 3u IPv6 19810 0t0 TCP localhost:pmcd (LISTEN) 2023-07-29T14:21:36.995 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 4u unix 0xffff89b9c7b46a40 0t0 19811 /run/pcp/pmcd.socket type=STREAM (LISTEN) 2023-07-29T14:21:36.995 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 5u unix 0xffff89b9c82f72c0 0t0 20947 type=STREAM (CONNECTED) 2023-07-29T14:21:36.995 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 6w FIFO 0,13 0t0 19817 pipe 2023-07-29T14:21:36.995 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 7r FIFO 0,13 0t0 19818 pipe 2023-07-29T14:21:36.995 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 8u unix 0xffff89b9c82f0000 0t0 20948 type=STREAM (CONNECTED) 2023-07-29T14:21:36.995 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 9w FIFO 0,13 0t0 21894 pipe 2023-07-29T14:21:36.995 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 10r FIFO 0,13 0t0 21895 pipe 2023-07-29T14:21:36.995 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 11w FIFO 0,13 0t0 19833 pipe 2023-07-29T14:21:36.996 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 12r FIFO 0,13 0t0 19834 pipe 2023-07-29T14:21:36.996 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 13w FIFO 0,13 0t0 19835 pipe 2023-07-29T14:21:36.996 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 14r FIFO 0,13 0t0 19836 pipe 2023-07-29T14:21:36.996 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 15w FIFO 0,13 0t0 19838 pipe 2023-07-29T14:21:36.996 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 16r FIFO 0,13 0t0 19839 pipe 2023-07-29T14:21:36.996 INFO:teuthology.orchestra.run.smithi045.stdout:pmcd 1159 pcp 17u unix 0xffff89b9c7b461c0 0t0 16911 /run/pcp/pmcd.socket type=STREAM (CONNECTED) 2023-07-29T14:21:36.996 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root cwd DIR 8,1 4096 1610 /var/log/pcp/pmcd 2023-07-29T14:21:36.996 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:36.997 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root txt REG 8,1 57984 53942 /usr/libexec/pcp/pmdas/root/pmdaroot 2023-07-29T14:21:36.997 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:36.997 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:36.997 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:36.997 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:36.997 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:36.997 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:36.997 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:36.998 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:36.998 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:36.998 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:36.998 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:36.998 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:36.998 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:36.998 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:36.998 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:36.999 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:36.999 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:36.999 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:36.999 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:36.999 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 82624 9851 /usr/lib64/libavahi-client.so.3.2.9 2023-07-29T14:21:36.999 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:36.999 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:36.999 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 58464 9853 /usr/lib64/libavahi-common.so.3.5.4 2023-07-29T14:21:37.000 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.000 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.000 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 209408 11678 /usr/lib64/libuv.so.1.0.0 2023-07-29T14:21:37.000 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.000 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.000 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 32080 12530 /usr/lib64/libpcp_mmv.so.1 2023-07-29T14:21:37.000 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 758088 12526 /usr/lib64/libpcp.so.3 2023-07-29T14:21:37.001 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 107344 12531 /usr/lib64/libpcp_pmda.so.3 2023-07-29T14:21:37.001 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 531608 12533 /usr/lib64/libpcp_web.so.1 2023-07-29T14:21:37.001 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 686 53941 /usr/libexec/pcp/pmdas/root/help.pag 2023-07-29T14:21:37.001 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.001 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root mem REG 8,1 108 53940 /usr/libexec/pcp/pmdas/root/help.dir 2023-07-29T14:21:37.001 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root 0r FIFO 0,13 0t0 19817 pipe 2023-07-29T14:21:37.001 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root 1w FIFO 0,13 0t0 19818 pipe 2023-07-29T14:21:37.001 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root 2w REG 8,1 466 1625 /var/log/pcp/pmcd/root.log 2023-07-29T14:21:37.002 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root 3u unix 0xffff89b9c7b44840 0t0 19827 /var/lib/pcp/pmcd/root.socket type=STREAM (LISTEN) 2023-07-29T14:21:37.002 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root 4u unix 0xffff89b9c82f6600 0t0 19830 /var/lib/pcp/pmcd/root.socket type=STREAM (CONNECTED) 2023-07-29T14:21:37.002 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root 5u unix 0xffff89b9c82f6a40 0t0 19831 /var/lib/pcp/pmcd/root.socket type=STREAM (CONNECTED) 2023-07-29T14:21:37.002 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root 6u unix 0xffff89b9d821c840 0t0 19832 /var/lib/pcp/pmcd/root.socket type=STREAM (CONNECTED) 2023-07-29T14:21:37.002 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaroot 1163 root 7u unix 0xffff89b9c7b8e600 0t0 19837 /var/lib/pcp/pmcd/root.socket type=STREAM (CONNECTED) 2023-07-29T14:21:37.002 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root cwd DIR 8,1 4096 1610 /var/log/pcp/pmcd 2023-07-29T14:21:37.002 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.002 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root txt REG 8,1 184184 53934 /usr/libexec/pcp/pmdas/proc/pmdaproc 2023-07-29T14:21:37.003 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 31640 53932 /usr/libexec/pcp/pmdas/proc/help.pag 2023-07-29T14:21:37.003 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.003 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.003 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.003 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.003 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.003 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.003 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.004 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.004 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.004 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.004 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:37.004 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.004 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.004 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.004 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.005 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.005 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.005 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.005 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.005 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.005 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.005 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.005 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 82624 9851 /usr/lib64/libavahi-client.so.3.2.9 2023-07-29T14:21:37.006 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.006 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 58464 9853 /usr/lib64/libavahi-common.so.3.5.4 2023-07-29T14:21:37.006 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.006 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.006 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 758088 12526 /usr/lib64/libpcp.so.3 2023-07-29T14:21:37.006 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 107344 12531 /usr/lib64/libpcp_pmda.so.3 2023-07-29T14:21:37.006 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.007 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root mem REG 8,1 3300 53931 /usr/libexec/pcp/pmdas/proc/help.dir 2023-07-29T14:21:37.007 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root 0r FIFO 0,13 0t0 21894 pipe 2023-07-29T14:21:37.007 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root 1w FIFO 0,13 0t0 21895 pipe 2023-07-29T14:21:37.007 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root 2w REG 8,1 64 54369 /var/log/pcp/pmcd/proc.log 2023-07-29T14:21:37.007 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaproc 1214 root 3u unix 0xffff89b9d821aa80 0t0 1921 type=STREAM (CONNECTED) 2023-07-29T14:21:37.007 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root cwd DIR 8,1 4096 1610 /var/log/pcp/pmcd 2023-07-29T14:21:37.007 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.007 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root txt REG 8,1 53424 53953 /usr/libexec/pcp/pmdas/xfs/pmdaxfs 2023-07-29T14:21:37.008 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 27178 53951 /usr/libexec/pcp/pmdas/xfs/help.pag 2023-07-29T14:21:37.008 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.008 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.008 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.008 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.008 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.008 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.008 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.009 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.009 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.009 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.009 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:37.009 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.009 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.009 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.009 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.010 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.010 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.010 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.010 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.010 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.010 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.010 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.010 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.011 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 82624 9851 /usr/lib64/libavahi-client.so.3.2.9 2023-07-29T14:21:37.011 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 58464 9853 /usr/lib64/libavahi-common.so.3.5.4 2023-07-29T14:21:37.011 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.011 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.011 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 758088 12526 /usr/lib64/libpcp.so.3 2023-07-29T14:21:37.011 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 107344 12531 /usr/lib64/libpcp_pmda.so.3 2023-07-29T14:21:37.011 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.011 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root mem REG 8,1 2496 53950 /usr/libexec/pcp/pmdas/xfs/help.dir 2023-07-29T14:21:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root 0r FIFO 0,13 0t0 19833 pipe 2023-07-29T14:21:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root 1w FIFO 0,13 0t0 19834 pipe 2023-07-29T14:21:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:pmdaxfs 1223 root 2w REG 8,1 63 54370 /var/log/pcp/pmcd/xfs.log 2023-07-29T14:21:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root cwd DIR 8,1 4096 1610 /var/log/pcp/pmcd 2023-07-29T14:21:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root txt REG 8,1 271456 53904 /usr/libexec/pcp/pmdas/linux/pmdalinux 2023-07-29T14:21:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 156737 53902 /usr/libexec/pcp/pmdas/linux/help.pag 2023-07-29T14:21:37.013 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.013 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.013 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.013 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.013 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.013 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.013 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.013 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.014 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.014 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.014 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:37.014 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.014 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.014 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.014 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.014 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.015 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.015 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.015 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.015 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.015 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.015 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 82624 9851 /usr/lib64/libavahi-client.so.3.2.9 2023-07-29T14:21:37.015 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.015 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.016 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.016 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.016 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 58464 9853 /usr/lib64/libavahi-common.so.3.5.4 2023-07-29T14:21:37.016 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 758088 12526 /usr/lib64/libpcp.so.3 2023-07-29T14:21:37.016 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 107344 12531 /usr/lib64/libpcp_pmda.so.3 2023-07-29T14:21:37.016 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 16428 53901 /usr/libexec/pcp/pmdas/linux/help.dir 2023-07-29T14:21:37.016 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.017 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root 0r FIFO 0,13 0t0 19835 pipe 2023-07-29T14:21:37.017 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root 1w FIFO 0,13 0t0 19836 pipe 2023-07-29T14:21:37.017 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root 2w REG 8,1 285 54371 /var/log/pcp/pmcd/linux.log 2023-07-29T14:21:37.017 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root 3u unix 0xffff89b9c7b88000 0t0 21902 type=STREAM (CONNECTED) 2023-07-29T14:21:37.017 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root 4r REG 0,20 0 4026532033 /proc/stat 2023-07-29T14:21:37.017 INFO:teuthology.orchestra.run.smithi045.stdout:pmdalinux 1224 root 5u sock 0,8 0t0 1943 protocol: UDP 2023-07-29T14:21:37.017 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root cwd DIR 8,1 4096 1610 /var/log/pcp/pmcd 2023-07-29T14:21:37.017 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.018 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root txt REG 8,1 29864 53895 /usr/libexec/pcp/pmdas/kvm/pmdakvm 2023-07-29T14:21:37.018 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.018 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.018 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.018 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.018 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.018 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.018 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.019 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.019 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.019 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.019 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:37.019 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.019 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.019 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.019 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.020 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.020 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.020 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.020 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.020 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.020 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.020 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.020 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.021 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.021 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 82624 9851 /usr/lib64/libavahi-client.so.3.2.9 2023-07-29T14:21:37.021 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 58464 9853 /usr/lib64/libavahi-common.so.3.5.4 2023-07-29T14:21:37.021 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.021 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 758088 12526 /usr/lib64/libpcp.so.3 2023-07-29T14:21:37.021 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 107344 12531 /usr/lib64/libpcp_pmda.so.3 2023-07-29T14:21:37.021 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 3147 53893 /usr/libexec/pcp/pmdas/kvm/help.pag 2023-07-29T14:21:37.022 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.022 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root mem REG 8,1 540 53892 /usr/libexec/pcp/pmdas/kvm/help.dir 2023-07-29T14:21:37.022 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root 0r FIFO 0,13 0t0 19838 pipe 2023-07-29T14:21:37.022 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root 1w FIFO 0,13 0t0 19839 pipe 2023-07-29T14:21:37.022 INFO:teuthology.orchestra.run.smithi045.stdout:pmdakvm 1232 root 2w REG 8,1 63 54372 /var/log/pcp/pmcd/kvm.log 2023-07-29T14:21:37.022 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp cwd DIR 8,1 4096 263236 /var/log/pcp/pmlogger/smithi045 2023-07-29T14:21:37.022 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.022 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp txt REG 8,1 157480 12554 /usr/bin/pmlogger 2023-07-29T14:21:37.023 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.023 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.023 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.023 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.023 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.023 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.023 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.023 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.024 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:37.024 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.024 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.024 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.024 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.024 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.024 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.024 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.025 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.025 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.025 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.025 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.025 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.025 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.025 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.026 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.026 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 82624 9851 /usr/lib64/libavahi-client.so.3.2.9 2023-07-29T14:21:37.026 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 58464 9853 /usr/lib64/libavahi-common.so.3.5.4 2023-07-29T14:21:37.026 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.026 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 758088 12526 /usr/lib64/libpcp.so.3 2023-07-29T14:21:37.026 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 27672 12527 /usr/lib64/libpcp_archive.so.1 2023-07-29T14:21:37.026 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.026 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp 0r FIFO 0,13 0t0 22988 pipe 2023-07-29T14:21:37.027 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp 1w REG 8,1 0 263663 /var/tmp/pmlogger_check.skJHIbucE/out (deleted) 2023-07-29T14:21:37.027 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp 2w REG 8,1 26469 263666 /var/log/pcp/pmlogger/smithi045/pmlogger.log 2023-07-29T14:21:37.027 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp 3u unix 0xffff89b9c7b43fc0 0t0 19884 type=STREAM (CONNECTED) 2023-07-29T14:21:37.027 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp 4w REG 8,1 812 263668 /var/log/pcp/pmlogger/smithi045/20230729.13.09.index 2023-07-29T14:21:37.027 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp 5w REG 8,1 353838 263669 /var/log/pcp/pmlogger/smithi045/20230729.13.09.meta 2023-07-29T14:21:37.027 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp 6w REG 8,1 2572052 263670 /var/log/pcp/pmlogger/smithi045/20230729.13.09.0 2023-07-29T14:21:37.027 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp 7u IPv4 1938 0t0 TCP localhost:dey-sapi (LISTEN) 2023-07-29T14:21:37.027 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp 8u IPv6 1939 0t0 TCP localhost:dey-sapi (LISTEN) 2023-07-29T14:21:37.028 INFO:teuthology.orchestra.run.smithi045.stdout:pmlogger 1520 pcp 9u unix 0xffff89b9d8218440 0t0 1940 /run/pcp/pmlogger.1520.socket type=STREAM (LISTEN) 2023-07-29T14:21:37.028 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp cwd DIR 8,1 4096 1233 /var/lib/pcp 2023-07-29T14:21:37.028 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.028 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp txt REG 8,1 15504 47000 /usr/libexec/pcp/bin/pmsleep 2023-07-29T14:21:37.028 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.028 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.028 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.028 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.029 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.029 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.029 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.029 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.029 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 343744 9543 /usr/lib64/libdbus-1.so.3.19.13 2023-07-29T14:21:37.029 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.029 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.030 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.030 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.030 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.030 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.030 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.030 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.030 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.030 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.031 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.031 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.031 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.031 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.031 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 82624 9851 /usr/lib64/libavahi-client.so.3.2.9 2023-07-29T14:21:37.031 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 58464 9853 /usr/lib64/libavahi-common.so.3.5.4 2023-07-29T14:21:37.031 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.031 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.032 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 758088 12526 /usr/lib64/libpcp.so.3 2023-07-29T14:21:37.032 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.032 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.032 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp 1u unix 0xffff89b9c7b45940 0t0 19909 type=STREAM (CONNECTED) 2023-07-29T14:21:37.032 INFO:teuthology.orchestra.run.smithi045.stdout:pmpause 1603 pcp 2u unix 0xffff89b9c7b45940 0t0 19909 type=STREAM (CONNECTED) 2023-07-29T14:21:37.032 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1851 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.032 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1851 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.032 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1851 root txt REG 8,1 58080 9631 /usr/sbin/agetty 2023-07-29T14:21:37.033 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1851 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.033 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1851 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:37.033 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1851 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:37.033 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1851 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.033 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1851 root 0u CHR 4,1 0t0 20 /dev/tty1 2023-07-29T14:21:37.033 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1851 root 1u CHR 4,1 0t0 20 /dev/tty1 2023-07-29T14:21:37.033 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1851 root 2u CHR 4,1 0t0 20 /dev/tty1 2023-07-29T14:21:37.034 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1851 root 4r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:37.034 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.034 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.034 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root txt REG 8,1 58080 9631 /usr/sbin/agetty 2023-07-29T14:21:37.034 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:37.034 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.034 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:37.034 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:37.035 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:37.035 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:37.035 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:37.035 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:37.035 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:37.035 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:37.035 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:37.035 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:37.036 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:37.036 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.036 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:37.036 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root 0u CHR 4,65 0t0 114 /dev/ttyS1 2023-07-29T14:21:37.036 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root 1u CHR 4,65 0t0 114 /dev/ttyS1 2023-07-29T14:21:37.036 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root 2u CHR 4,65 0t0 114 /dev/ttyS1 2023-07-29T14:21:37.036 INFO:teuthology.orchestra.run.smithi045.stdout:agetty 1853 root 4r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:37.036 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.037 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.037 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu txt REG 8,1 102120 16840 /usr/lib/systemd/systemd 2023-07-29T14:21:37.037 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu DEL REG 8,1 111 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-07-29T14:21:37.037 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.037 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.037 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.037 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.038 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:37.038 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.038 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu DEL REG 8,1 28863 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-07-29T14:21:37.038 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:37.038 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.038 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.038 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.038 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.039 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 3640680 43416 /usr/lib64/systemd/libsystemd-shared-252.so 2023-07-29T14:21:37.039 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 28552 9218 /usr/lib64/libattr.so.1.1.2501 2023-07-29T14:21:37.039 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:37.039 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:37.039 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.039 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.039 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.039 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.040 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 41120 9220 /usr/lib64/libacl.so.1.1.2301 2023-07-29T14:21:37.040 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:37.040 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.040 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 107984 9595 /usr/lib64/libkmod.so.2.3.6 2023-07-29T14:21:37.040 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:37.040 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:37.040 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 127680 9386 /usr/lib64/libseccomp.so.2.5.2 2023-07-29T14:21:37.040 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.041 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 2112568 43415 /usr/lib64/systemd/libsystemd-core-252.so 2023-07-29T14:21:37.041 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.041 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2023-07-29T14:21:37.041 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.041 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 1u unix 0xffff89b9d8162ec0 0t0 24208 type=STREAM (CONNECTED) 2023-07-29T14:21:37.041 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 2u unix 0xffff89b9d8162ec0 0t0 24208 type=STREAM (CONNECTED) 2023-07-29T14:21:37.041 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 3u unix 0xffff89b9c8e45d80 0t0 24235 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.042 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 4u a_inode 0,14 0 2079 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2023-07-29T14:21:37.042 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 5u a_inode 0,14 0 2079 [signalfd] 2023-07-29T14:21:37.042 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 6r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:37.042 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 7r DIR 0,25 0 3493 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2023-07-29T14:21:37.042 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 8u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:37.042 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 9u a_inode 0,14 0 2079 [eventpoll:10,13] 2023-07-29T14:21:37.042 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 10r REG 0,20 0 24917 /proc/5469/mountinfo 2023-07-29T14:21:37.042 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 11r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:37.043 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 12u unix 0xffff89b9c8e7c400 0t0 24936 /run/user/1000/bus type=STREAM (LISTEN) 2023-07-29T14:21:37.043 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 13r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:37.043 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 14r REG 0,20 0 4026532076 /proc/swaps 2023-07-29T14:21:37.043 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 15u netlink 0t0 24918 KOBJECT_UEVENT 2023-07-29T14:21:37.043 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 16u unix 0xffff89b9c8e82ec0 0t0 24923 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2023-07-29T14:21:37.043 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 17u unix 0xffff89b9c8e83740 0t0 24924 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.043 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 18u unix 0xffff89b9c8e80880 0t0 24925 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.043 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 19u unix 0xffff89b9c8e78880 0t0 24926 /run/user/1000/systemd/private type=STREAM (LISTEN) 2023-07-29T14:21:37.044 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 20u unix 0xffff89b9c8e7cc80 0t0 24928 type=STREAM (CONNECTED) 2023-07-29T14:21:37.044 INFO:teuthology.orchestra.run.smithi045.stdout:systemd 5469 ubuntu 21u a_inode 0,14 0 2079 [timerfd] 2023-07-29T14:21:37.044 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.044 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.044 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu txt REG 8,1 102120 16840 /usr/lib/systemd/systemd 2023-07-29T14:21:37.044 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 357408 9535 /usr/lib64/libnss_systemd.so.2 2023-07-29T14:21:37.044 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 514272 27100 /usr/lib64/security/pam_systemd.so 2023-07-29T14:21:37.044 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu DEL REG 8,1 2101 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-07-29T14:21:37.045 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.045 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.045 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.045 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.045 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 14928 26881 /usr/lib64/security/pam_deny.so 2023-07-29T14:21:37.045 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 19600 10382 /usr/lib64/libpam_misc.so.0.82.1 2023-07-29T14:21:37.045 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 19400 26916 /usr/lib64/security/pam_succeed_if.so 2023-07-29T14:21:37.046 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 27536 26895 /usr/lib64/security/pam_limits.so 2023-07-29T14:21:37.046 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 15352 26893 /usr/lib64/security/pam_keyinit.so 2023-07-29T14:21:37.046 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 15240 26898 /usr/lib64/security/pam_loginuid.so 2023-07-29T14:21:37.046 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 27616 26910 /usr/lib64/security/pam_selinux.so 2023-07-29T14:21:37.046 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 56712 26921 /usr/lib64/security/pam_unix.so 2023-07-29T14:21:37.046 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:37.046 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.046 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.047 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:37.047 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.047 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 28552 9218 /usr/lib64/libattr.so.1.1.2501 2023-07-29T14:21:37.047 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:37.047 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:37.047 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.047 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.047 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.048 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.048 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.048 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 3640680 43416 /usr/lib64/systemd/libsystemd-shared-252.so 2023-07-29T14:21:37.048 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.048 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 41120 9220 /usr/lib64/libacl.so.1.1.2301 2023-07-29T14:21:37.048 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 283152 9571 /usr/lib64/libmount.so.1.1.0 2023-07-29T14:21:37.048 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.048 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 107984 9595 /usr/lib64/libkmod.so.2.3.6 2023-07-29T14:21:37.049 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:37.049 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:37.049 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 127680 9386 /usr/lib64/libseccomp.so.2.5.2 2023-07-29T14:21:37.049 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.049 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu DEL REG 8,1 2102 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-07-29T14:21:37.049 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 2112568 43415 /usr/lib64/systemd/libsystemd-core-252.so 2023-07-29T14:21:37.049 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.049 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2023-07-29T14:21:37.050 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.050 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu 1u unix 0xffff89b9d8162ec0 0t0 24208 type=STREAM (CONNECTED) 2023-07-29T14:21:37.050 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu 2u unix 0xffff89b9d8162ec0 0t0 24208 type=STREAM (CONNECTED) 2023-07-29T14:21:37.050 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu 3u a_inode 0,14 0 2079 [eventfd:9] 2023-07-29T14:21:37.050 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu 4u a_inode 0,14 0 2079 [eventfd:10] 2023-07-29T14:21:37.050 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu 6w FIFO 0,13 0t0 24212 pipe 2023-07-29T14:21:37.050 INFO:teuthology.orchestra.run.smithi045.stdout:(sd-pam) 5471 ubuntu 7u unix 0xffff89b9c8e43300 0t0 24227 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.051 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.051 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.051 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root txt REG 8,1 769552 11652 /usr/sbin/rsyslogd 2023-07-29T14:21:37.051 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 0,24 33554432 58 /run/log/journal/e13cba1d25f14c51a1d04da8db823a4e/system.journal 2023-07-29T14:21:37.051 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 33584 11622 /usr/lib64/rsyslog/imjournal.so 2023-07-29T14:21:37.051 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 42248 11629 /usr/lib64/rsyslog/imuxsock.so 2023-07-29T14:21:37.051 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.051 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.052 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.052 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.052 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.052 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 32808 11630 /usr/lib64/rsyslog/lmnet.so 2023-07-29T14:21:37.052 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.052 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.052 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.052 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:37.053 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 32192 9127 /usr/lib64/libuuid.so.1.3.0 2023-07-29T14:21:37.053 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.053 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 53472 11616 /usr/lib64/libfastjson.so.4.3.0 2023-07-29T14:21:37.053 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 16032 11614 /usr/lib64/libestr.so.0.0.0 2023-07-29T14:21:37.053 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.053 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.053 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.053 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root 1w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.054 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root 2w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.054 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root 3r CHR 1,9 0t0 9 /dev/urandom 2023-07-29T14:21:37.054 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root 4u unix 0xffff89b9c8cde600 0t0 26763 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.054 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root 5w REG 8,1 674509 1670 /var/log/secure 2023-07-29T14:21:37.054 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root 6r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:37.054 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root 7w REG 8,1 11443050 1669 /var/log/messages 2023-07-29T14:21:37.054 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root 8w REG 8,1 3832 1712 /var/log/cron 2023-07-29T14:21:37.054 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 root 9r REG 0,24 33554432 58 /run/log/journal/e13cba1d25f14c51a1d04da8db823a4e/system.journal 2023-07-29T14:21:37.055 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.055 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.055 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root txt REG 8,1 769552 11652 /usr/sbin/rsyslogd 2023-07-29T14:21:37.055 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 0,24 33554432 58 /run/log/journal/e13cba1d25f14c51a1d04da8db823a4e/system.journal 2023-07-29T14:21:37.055 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 33584 11622 /usr/lib64/rsyslog/imjournal.so 2023-07-29T14:21:37.055 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 42248 11629 /usr/lib64/rsyslog/imuxsock.so 2023-07-29T14:21:37.055 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.056 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.056 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.056 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.056 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.056 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 32808 11630 /usr/lib64/rsyslog/lmnet.so 2023-07-29T14:21:37.056 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.056 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.056 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.057 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:37.057 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 32192 9127 /usr/lib64/libuuid.so.1.3.0 2023-07-29T14:21:37.057 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.057 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 53472 11616 /usr/lib64/libfastjson.so.4.3.0 2023-07-29T14:21:37.057 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 16032 11614 /usr/lib64/libestr.so.0.0.0 2023-07-29T14:21:37.057 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.057 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.057 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.058 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root 1w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.058 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root 2w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.058 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root 3r CHR 1,9 0t0 9 /dev/urandom 2023-07-29T14:21:37.058 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root 4u unix 0xffff89b9c8cde600 0t0 26763 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.058 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root 5w REG 8,1 674509 1670 /var/log/secure 2023-07-29T14:21:37.058 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root 6r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:37.058 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root 7w REG 8,1 11443050 1669 /var/log/messages 2023-07-29T14:21:37.058 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root 8w REG 8,1 3832 1712 /var/log/cron 2023-07-29T14:21:37.059 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5730 in:imjour root 9r REG 0,24 33554432 58 /run/log/journal/e13cba1d25f14c51a1d04da8db823a4e/system.journal 2023-07-29T14:21:37.059 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.059 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.059 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root txt REG 8,1 769552 11652 /usr/sbin/rsyslogd 2023-07-29T14:21:37.059 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 0,24 33554432 58 /run/log/journal/e13cba1d25f14c51a1d04da8db823a4e/system.journal 2023-07-29T14:21:37.059 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 33584 11622 /usr/lib64/rsyslog/imjournal.so 2023-07-29T14:21:37.059 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 42248 11629 /usr/lib64/rsyslog/imuxsock.so 2023-07-29T14:21:37.059 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.060 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.060 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.060 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.060 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.060 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 32808 11630 /usr/lib64/rsyslog/lmnet.so 2023-07-29T14:21:37.060 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.060 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.060 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.061 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:37.061 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 32192 9127 /usr/lib64/libuuid.so.1.3.0 2023-07-29T14:21:37.061 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.061 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 53472 11616 /usr/lib64/libfastjson.so.4.3.0 2023-07-29T14:21:37.061 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 16032 11614 /usr/lib64/libestr.so.0.0.0 2023-07-29T14:21:37.061 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.061 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.062 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.062 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root 1w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.062 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root 2w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.062 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root 3r CHR 1,9 0t0 9 /dev/urandom 2023-07-29T14:21:37.062 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root 4u unix 0xffff89b9c8cde600 0t0 26763 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.062 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root 5w REG 8,1 674509 1670 /var/log/secure 2023-07-29T14:21:37.062 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root 6r a_inode 0,14 0 2079 inotify 2023-07-29T14:21:37.062 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root 7w REG 8,1 11443050 1669 /var/log/messages 2023-07-29T14:21:37.063 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root 8w REG 8,1 3832 1712 /var/log/cron 2023-07-29T14:21:37.063 INFO:teuthology.orchestra.run.smithi045.stdout:rsyslogd 5728 5731 rs:main root 9r REG 0,24 33554432 58 /run/log/journal/e13cba1d25f14c51a1d04da8db823a4e/system.journal 2023-07-29T14:21:37.063 INFO:teuthology.orchestra.run.smithi045.stdout:tls-strp 58231 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.063 INFO:teuthology.orchestra.run.smithi045.stdout:tls-strp 58231 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.063 INFO:teuthology.orchestra.run.smithi045.stdout:tls-strp 58231 root txt unknown /proc/58231/exe 2023-07-29T14:21:37.063 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.063 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.063 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root txt REG 8,1 622048 11938 /usr/sbin/smartd 2023-07-29T14:21:37.064 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.064 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root DEL REG 8,1 9298 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.064 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.064 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.064 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.064 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.064 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.064 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.065 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.065 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.065 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.065 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.065 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.065 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:37.065 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.065 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.066 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.066 INFO:teuthology.orchestra.run.smithi045.stdout:smartd 58574 root 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.066 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68085 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.066 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68085 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.066 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68085 root txt unknown /proc/68085/exe 2023-07-29T14:21:37.066 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68205 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.066 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68205 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.067 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68205 root txt unknown /proc/68205/exe 2023-07-29T14:21:37.067 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68325 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.067 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68325 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.067 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68325 root txt unknown /proc/68325/exe 2023-07-29T14:21:37.067 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68445 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.067 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68445 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.067 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68445 root txt unknown /proc/68445/exe 2023-07-29T14:21:37.067 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68565 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.068 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68565 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.068 INFO:teuthology.orchestra.run.smithi045.stdout:kdmflush/ 68565 root txt unknown /proc/68565/exe 2023-07-29T14:21:37.068 INFO:teuthology.orchestra.run.smithi045.stdout:xfsalloc 69458 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.068 INFO:teuthology.orchestra.run.smithi045.stdout:xfsalloc 69458 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.068 INFO:teuthology.orchestra.run.smithi045.stdout:xfsalloc 69458 root txt unknown /proc/69458/exe 2023-07-29T14:21:37.068 INFO:teuthology.orchestra.run.smithi045.stdout:xfs_mru_c 69459 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.068 INFO:teuthology.orchestra.run.smithi045.stdout:xfs_mru_c 69459 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.068 INFO:teuthology.orchestra.run.smithi045.stdout:xfs_mru_c 69459 root txt unknown /proc/69459/exe 2023-07-29T14:21:37.069 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-buf/d 69460 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.069 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-buf/d 69460 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.069 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-buf/d 69460 root txt unknown /proc/69460/exe 2023-07-29T14:21:37.069 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-conv/ 69461 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.069 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-conv/ 69461 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.069 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-conv/ 69461 root txt unknown /proc/69461/exe 2023-07-29T14:21:37.069 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-recla 69462 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.069 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-recla 69462 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.070 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-recla 69462 root txt unknown /proc/69462/exe 2023-07-29T14:21:37.070 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-block 69463 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.070 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-block 69463 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.070 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-block 69463 root txt unknown /proc/69463/exe 2023-07-29T14:21:37.070 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-inode 69464 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.070 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-inode 69464 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.070 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-inode 69464 root txt unknown /proc/69464/exe 2023-07-29T14:21:37.070 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-log/d 69465 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.071 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-log/d 69465 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.071 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-log/d 69465 root txt unknown /proc/69465/exe 2023-07-29T14:21:37.071 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-cil/d 69466 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.071 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-cil/d 69466 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.071 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-cil/d 69466 root txt unknown /proc/69466/exe 2023-07-29T14:21:37.071 INFO:teuthology.orchestra.run.smithi045.stdout:xfsaild/d 69467 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.071 INFO:teuthology.orchestra.run.smithi045.stdout:xfsaild/d 69467 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.071 INFO:teuthology.orchestra.run.smithi045.stdout:xfsaild/d 69467 root txt unknown /proc/69467/exe 2023-07-29T14:21:37.072 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 72478 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.072 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 72478 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.072 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/1 72478 root txt unknown /proc/72478/exe 2023-07-29T14:21:37.072 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.072 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.072 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root txt REG 8,1 956448 11769 /usr/sbin/sshd 2023-07-29T14:21:37.072 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 357408 9535 /usr/lib64/libnss_systemd.so.2 2023-07-29T14:21:37.073 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 19512 26894 /usr/lib64/security/pam_lastlog.so 2023-07-29T14:21:37.073 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 15184 26920 /usr/lib64/security/pam_umask.so 2023-07-29T14:21:37.073 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:37.073 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:37.073 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 19600 10382 /usr/lib64/libpam_misc.so.0.82.1 2023-07-29T14:21:37.073 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 19400 26916 /usr/lib64/security/pam_succeed_if.so 2023-07-29T14:21:37.073 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 514272 27100 /usr/lib64/security/pam_systemd.so 2023-07-29T14:21:37.073 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 27536 26895 /usr/lib64/security/pam_limits.so 2023-07-29T14:21:37.074 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 15264 26901 /usr/lib64/security/pam_motd.so 2023-07-29T14:21:37.074 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 15352 26893 /usr/lib64/security/pam_keyinit.so 2023-07-29T14:21:37.074 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 44152 26902 /usr/lib64/security/pam_namespace.so 2023-07-29T14:21:37.074 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 27616 26910 /usr/lib64/security/pam_selinux.so 2023-07-29T14:21:37.074 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 48392 9642 /usr/lib64/libcrack.so.2.9.0 2023-07-29T14:21:37.074 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 32608 10375 /usr/lib64/libpwquality.so.1.0.2 2023-07-29T14:21:37.074 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 15240 26898 /usr/lib64/security/pam_loginuid.so 2023-07-29T14:21:37.074 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 15840 26832 /usr/lib64/security/pam_pwquality.so 2023-07-29T14:21:37.075 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 15232 26903 /usr/lib64/security/pam_nologin.so 2023-07-29T14:21:37.075 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 19368 26912 /usr/lib64/security/pam_sepermit.so 2023-07-29T14:21:37.075 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 14928 26881 /usr/lib64/security/pam_deny.so 2023-07-29T14:21:37.075 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 56712 26921 /usr/lib64/security/pam_unix.so 2023-07-29T14:21:37.075 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 15184 26885 /usr/lib64/security/pam_faildelay.so 2023-07-29T14:21:37.075 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.075 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.075 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.076 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.076 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.076 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.076 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 1288504 2548 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.076 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.076 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.076 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.076 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.078 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.078 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.078 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.078 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.078 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.078 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:37.078 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:37.079 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 19464 26883 /usr/lib64/security/pam_env.so 2023-07-29T14:21:37.079 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.079 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-07-29T14:21:37.079 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.079 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.079 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.079 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root 3u unix 0xffff89b9c44a2a80 0t0 161097 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.079 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root 4u IPv4 149450 0t0 TCP smithi045.front.sepia.ceph.com:ssh->teuthology.front.sepia.ceph.com:54918 (ESTABLISHED) 2023-07-29T14:21:37.080 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root 6u unix 0xffff89b9c449ea40 0t0 161072 type=STREAM (CONNECTED) 2023-07-29T14:21:37.080 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root 7w FIFO 0,24 0t0 1768 /run/systemd/sessions/4.ref 2023-07-29T14:21:37.080 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73566 root 8u unix 0xffff89b9c449cc80 0t0 161101 type=STREAM (CONNECTED) 2023-07-29T14:21:37.080 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.080 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.080 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu txt REG 8,1 956448 11769 /usr/sbin/sshd 2023-07-29T14:21:37.080 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 357408 9535 /usr/lib64/libnss_systemd.so.2 2023-07-29T14:21:37.080 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 19512 26894 /usr/lib64/security/pam_lastlog.so 2023-07-29T14:21:37.081 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 15184 26920 /usr/lib64/security/pam_umask.so 2023-07-29T14:21:37.081 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:37.081 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:37.081 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 19600 10382 /usr/lib64/libpam_misc.so.0.82.1 2023-07-29T14:21:37.081 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 19400 26916 /usr/lib64/security/pam_succeed_if.so 2023-07-29T14:21:37.081 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 514272 27100 /usr/lib64/security/pam_systemd.so 2023-07-29T14:21:37.081 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 27536 26895 /usr/lib64/security/pam_limits.so 2023-07-29T14:21:37.081 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 15264 26901 /usr/lib64/security/pam_motd.so 2023-07-29T14:21:37.082 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 15352 26893 /usr/lib64/security/pam_keyinit.so 2023-07-29T14:21:37.082 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 44152 26902 /usr/lib64/security/pam_namespace.so 2023-07-29T14:21:37.082 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 27616 26910 /usr/lib64/security/pam_selinux.so 2023-07-29T14:21:37.082 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 48392 9642 /usr/lib64/libcrack.so.2.9.0 2023-07-29T14:21:37.082 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 32608 10375 /usr/lib64/libpwquality.so.1.0.2 2023-07-29T14:21:37.082 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 15240 26898 /usr/lib64/security/pam_loginuid.so 2023-07-29T14:21:37.082 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 15840 26832 /usr/lib64/security/pam_pwquality.so 2023-07-29T14:21:37.083 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 15232 26903 /usr/lib64/security/pam_nologin.so 2023-07-29T14:21:37.083 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 19368 26912 /usr/lib64/security/pam_sepermit.so 2023-07-29T14:21:37.083 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 14928 26881 /usr/lib64/security/pam_deny.so 2023-07-29T14:21:37.083 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 56712 26921 /usr/lib64/security/pam_unix.so 2023-07-29T14:21:37.083 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 15184 26885 /usr/lib64/security/pam_faildelay.so 2023-07-29T14:21:37.083 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 153600 9223 /usr/lib64/libgpg-error.so.0.32.0 2023-07-29T14:21:37.083 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.083 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.084 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.084 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.084 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 882376 9113 /usr/lib64/libzstd.so.1.5.1 2023-07-29T14:21:37.084 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 1288504 2548 /usr/lib64/libgcrypt.so.20.4.0 2023-07-29T14:21:37.084 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.084 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.084 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.084 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.085 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.085 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.085 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.085 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.085 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:37.085 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:37.085 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.085 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.086 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.086 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.086 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.086 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.086 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 915720 9537 /usr/lib64/libsystemd.so.0.35.0 2023-07-29T14:21:37.086 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:37.086 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:37.087 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 19464 26883 /usr/lib64/security/pam_env.so 2023-07-29T14:21:37.087 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.087 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2023-07-29T14:21:37.087 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 0u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.087 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 1u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.087 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.087 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 3u unix 0xffff89b9c44a2a80 0t0 161097 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.087 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 4u IPv4 149450 0t0 TCP smithi045.front.sepia.ceph.com:ssh->teuthology.front.sepia.ceph.com:54918 (ESTABLISHED) 2023-07-29T14:21:37.088 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 5u unix 0xffff89b9c449d500 0t0 161100 type=STREAM (CONNECTED) 2023-07-29T14:21:37.088 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 6u unix 0xffff89b9c449ea40 0t0 161072 type=STREAM (CONNECTED) 2023-07-29T14:21:37.088 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 7w FIFO 0,24 0t0 1768 /run/systemd/sessions/4.ref 2023-07-29T14:21:37.088 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 11w FIFO 0,13 0t0 178953 pipe 2023-07-29T14:21:37.088 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 12r FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.088 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 14r FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.088 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 15r FIFO 0,13 0t0 199374 pipe 2023-07-29T14:21:37.088 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 17r FIFO 0,13 0t0 199375 pipe 2023-07-29T14:21:37.089 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 25w FIFO 0,13 0t0 185403 pipe 2023-07-29T14:21:37.089 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 27r FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.089 INFO:teuthology.orchestra.run.smithi045.stdout:sshd 73569 ubuntu 29r FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.089 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.089 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.089 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony txt REG 8,1 353408 11904 /usr/sbin/chronyd 2023-07-29T14:21:37.089 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 626152 9403 /usr/lib64/libhogweed.so.6.5 2023-07-29T14:21:37.089 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.090 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 340456 9405 /usr/lib64/libnettle.so.8.5 2023-07-29T14:21:37.090 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.090 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:36.713+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 210 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:37.107 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 2291592 9409 /usr/lib64/libgnutls.so.30.33.1 2023-07-29T14:21:37.107 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:37.108 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 94112 9315 /usr/lib64/libtasn1.so.6.6.0 2023-07-29T14:21:37.108 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.108 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:37.108 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 127680 9386 /usr/lib64/libseccomp.so.2.5.2 2023-07-29T14:21:37.108 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.108 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.108 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.109 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.109 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony 1w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.109 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony 2u CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.109 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony 3u unix 0xffff89b9c8317b40 0t0 158350 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.109 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony 5u IPv4 158355 0t0 UDP localhost:323 2023-07-29T14:21:37.109 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony 6u IPv6 158356 0t0 UDP localhost:323 2023-07-29T14:21:37.109 INFO:teuthology.orchestra.run.smithi045.stdout:chronyd 73629 chrony 7u unix 0xffff89b9c8314400 0t0 158357 /run/chrony/chronyd.sock type=DGRAM (UNCONNECTED) 2023-07-29T14:21:37.110 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.110 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.110 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph txt REG 8,1 15448 9913 /usr/bin/python3.9 2023-07-29T14:21:37.110 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.110 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 46560 32348 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.110 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 76792 9115 /usr/lib64/libbz2.so.1.0.8 2023-07-29T14:21:37.110 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.110 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.111 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 29232 32326 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.111 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 41752 32384 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.111 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 68000 32365 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.111 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 101216 32357 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.111 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 23800 32354 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.111 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 37592 32378 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.111 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 71616 32371 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.111 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:37.112 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 3474032 9915 /usr/lib64/libpython3.9.so.1.0 2023-07-29T14:21:37.112 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 20000 32370 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.112 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 28064 32345 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.112 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.112 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:37.112 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.112 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.113 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph 1u unix 0xffff89b9c6be2a80 0t0 163334 type=STREAM (CONNECTED) 2023-07-29T14:21:37.113 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-cras 73825 ceph 2u unix 0xffff89b9c6be2a80 0t0 163334 type=STREAM (CONNECTED) 2023-07-29T14:21:37.113 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 75511 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.113 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 75511 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.113 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 75511 root txt unknown /proc/75511/exe 2023-07-29T14:21:37.113 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 77622 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.113 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 77622 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.113 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 77622 root txt unknown /proc/77622/exe 2023-07-29T14:21:37.114 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-buf/d 79386 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.114 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-buf/d 79386 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.114 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-buf/d 79386 root txt unknown /proc/79386/exe 2023-07-29T14:21:37.114 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-conv/ 79387 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.114 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-conv/ 79387 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.114 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-conv/ 79387 root txt unknown /proc/79387/exe 2023-07-29T14:21:37.114 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-recla 79388 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.114 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-recla 79388 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.115 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-recla 79388 root txt unknown /proc/79388/exe 2023-07-29T14:21:37.115 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-block 79389 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.115 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-block 79389 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.115 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-block 79389 root txt unknown /proc/79389/exe 2023-07-29T14:21:37.115 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-inode 79390 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.115 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-inode 79390 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.115 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-inode 79390 root txt unknown /proc/79390/exe 2023-07-29T14:21:37.115 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-log/d 79391 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.116 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-log/d 79391 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.116 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-log/d 79391 root txt unknown /proc/79391/exe 2023-07-29T14:21:37.116 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-cil/d 79392 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.116 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-cil/d 79392 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.116 INFO:teuthology.orchestra.run.smithi045.stdout:xfs-cil/d 79392 root txt unknown /proc/79392/exe 2023-07-29T14:21:37.117 INFO:teuthology.orchestra.run.smithi045.stdout:xfsaild/d 79393 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.117 INFO:teuthology.orchestra.run.smithi045.stdout:xfsaild/d 79393 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.117 INFO:teuthology.orchestra.run.smithi045.stdout:xfsaild/d 79393 root txt unknown /proc/79393/exe 2023-07-29T14:21:37.117 INFO:teuthology.orchestra.run.smithi045.stdout:dio/dm-3 82960 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:37.117 INFO:teuthology.orchestra.run.smithi045.stdout:dio/dm-3 82960 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.117 INFO:teuthology.orchestra.run.smithi045.stdout:dio/dm-3 82960 root txt unknown /proc/82960/exe 2023-07-29T14:21:37.117 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.117 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.118 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root txt REG 8,1 185296 11174 /usr/bin/sudo 2023-07-29T14:21:37.118 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 357408 9535 /usr/lib64/libnss_systemd.so.2 2023-07-29T14:21:37.118 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:37.118 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.118 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:37.118 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.118 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 19600 10382 /usr/lib64/libpam_misc.so.0.82.1 2023-07-29T14:21:37.118 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 19400 26916 /usr/lib64/security/pam_succeed_if.so 2023-07-29T14:21:37.119 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 514272 27100 /usr/lib64/security/pam_systemd.so 2023-07-29T14:21:37.119 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 27536 26895 /usr/lib64/security/pam_limits.so 2023-07-29T14:21:37.119 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 48392 9642 /usr/lib64/libcrack.so.2.9.0 2023-07-29T14:21:37.119 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 32608 10375 /usr/lib64/libpwquality.so.1.0.2 2023-07-29T14:21:37.119 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 15352 26893 /usr/lib64/security/pam_keyinit.so 2023-07-29T14:21:37.119 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 15840 26832 /usr/lib64/security/pam_pwquality.so 2023-07-29T14:21:37.119 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.120 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.120 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.120 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 14928 26881 /usr/lib64/security/pam_deny.so 2023-07-29T14:21:37.120 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 56712 26921 /usr/lib64/security/pam_unix.so 2023-07-29T14:21:37.120 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.120 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.120 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.120 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.121 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.121 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:37.121 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 15184 26885 /usr/lib64/security/pam_faildelay.so 2023-07-29T14:21:37.121 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.121 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.121 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.121 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.121 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.122 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:37.122 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.122 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.122 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.122 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:37.122 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.122 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 19464 26883 /usr/lib64/security/pam_env.so 2023-07-29T14:21:37.122 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 472584 48691 /usr/libexec/sudo/sudoers.so 2023-07-29T14:21:37.123 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:37.123 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:37.123 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:37.123 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:37.123 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:37.123 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:37.123 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:37.123 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:37.124 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:37.124 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:37.124 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.124 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:37.124 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 119832 48687 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-07-29T14:21:37.125 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.125 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:37.125 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:37.125 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.125 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:37.125 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root 0r FIFO 0,13 0t0 178953 pipe 2023-07-29T14:21:37.125 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.125 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.126 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root 3r FIFO 0,13 0t0 180174 pipe 2023-07-29T14:21:37.126 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root 4w FIFO 0,13 0t0 180174 pipe 2023-07-29T14:21:37.126 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root 5u netlink 0t0 180179 AUDIT 2023-07-29T14:21:37.126 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 84572 root 6u unix 0xffff89b9c39d3300 0t0 180180 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.126 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.126 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.126 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root txt REG 8,1 15448 9913 /usr/bin/python3.9 2023-07-29T14:21:37.126 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.127 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 46560 32348 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.127 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 76792 9115 /usr/lib64/libbz2.so.1.0.8 2023-07-29T14:21:37.127 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 29232 32326 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.127 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 71616 32371 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.127 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.127 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 41752 32384 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.127 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.127 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 23800 32354 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.128 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 56416 32361 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.128 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.128 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 3474032 9915 /usr/lib64/libpython3.9.so.1.0 2023-07-29T14:21:37.128 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 20000 32370 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.128 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 37592 32378 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.128 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 23832 32369 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.128 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 28064 32345 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.129 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:37.129 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:37.129 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.129 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root 0r FIFO 0,13 0t0 178953 pipe 2023-07-29T14:21:37.129 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.129 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 84606 root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.129 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.129 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.130 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.130 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.130 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.130 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.130 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.130 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.130 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.130 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.131 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.131 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.131 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.131 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.131 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.131 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.131 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.131 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.132 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.132 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.132 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.132 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.132 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.133 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.133 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.133 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.133 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.133 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.133 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.133 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.133 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.134 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.134 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.134 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.134 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.134 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.134 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.134 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.134 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.135 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.135 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.135 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.135 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.135 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.135 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.135 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.135 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.136 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.136 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.136 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.136 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.136 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.136 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.136 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.136 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.137 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.137 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.137 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.137 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.137 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.137 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.137 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.138 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.138 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.138 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.138 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.138 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.138 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.138 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.138 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.139 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.139 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.139 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.139 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.139 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.139 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.139 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.139 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.140 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.140 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.140 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.140 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.140 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.140 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.141 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.141 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.141 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.141 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.141 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.141 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.142 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.142 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.142 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.142 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.142 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.142 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.142 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.143 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.143 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.143 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.143 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.143 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.143 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.143 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.143 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.144 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.144 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.144 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.144 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.144 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.144 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.144 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.144 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.145 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.145 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.145 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.145 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.145 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.145 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.145 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.146 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.146 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.146 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.146 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.146 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.146 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.146 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.146 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.147 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.147 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.147 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.147 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.147 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.147 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.147 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.147 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.148 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.148 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.148 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.148 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.148 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.148 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.148 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.148 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.149 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.149 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.149 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.149 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.149 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.149 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.150 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.150 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.150 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.150 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.150 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.150 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.150 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.150 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.151 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.151 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.151 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.151 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.151 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.151 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.151 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.152 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.152 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.152 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.152 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.152 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.152 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.152 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.152 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.153 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.153 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.153 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.153 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.153 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.153 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.153 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.153 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.154 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.154 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.154 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.154 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.154 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.154 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.154 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.154 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.155 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.155 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.155 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.155 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.155 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.155 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.155 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.156 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.156 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.156 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.156 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.156 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.156 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.156 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.156 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.157 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.157 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84620 log root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.157 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.157 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.157 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.157 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.157 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.158 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.158 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.158 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.158 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.158 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.158 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.158 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.159 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.159 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.159 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.159 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.159 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.159 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.159 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.159 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.160 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.160 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.160 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.160 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.160 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.160 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.160 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.161 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.161 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.161 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.161 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.161 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.161 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.161 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.161 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.162 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.162 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.162 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.162 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.162 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.162 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.162 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.163 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.163 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.163 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.163 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.163 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.163 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.163 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.163 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.164 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.164 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.164 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.164 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.164 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.164 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.164 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.165 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.165 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.165 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.165 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.165 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.165 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.165 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.165 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.166 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.166 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.166 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.166 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.166 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.166 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.167 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.167 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.167 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.167 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.167 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.167 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.167 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.168 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.168 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.168 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.168 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.168 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.168 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.168 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.168 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.169 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.169 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.169 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.169 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.169 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.169 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.169 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.170 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.170 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.170 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.170 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.170 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.170 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.170 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.170 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.171 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.171 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84621 service root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.171 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.171 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.171 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.171 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.171 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.171 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.172 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.172 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.172 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.172 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.172 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.172 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.172 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.173 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.173 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.173 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.173 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.173 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.173 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.173 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.173 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.174 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.174 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.174 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.174 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.174 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.174 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.175 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.175 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.175 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.175 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.175 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.175 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.175 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.175 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.176 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.176 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.176 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.176 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.176 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.176 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.176 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.176 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.177 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.177 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.177 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.177 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.177 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.177 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.177 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.177 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.178 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.178 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.178 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.178 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.178 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.178 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.178 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.178 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.179 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.179 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.179 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.179 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.179 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.179 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.179 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.180 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.180 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.180 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.180 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.180 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.180 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.180 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.180 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.181 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.181 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.181 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.181 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.181 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.181 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.181 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.181 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.182 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.182 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.182 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.182 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.182 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.182 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.182 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.183 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.183 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.183 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.183 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.183 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.183 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.183 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.184 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.184 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.184 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.184 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.184 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.184 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.184 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84622 admin_soc root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.184 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.185 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.185 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.185 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.185 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.185 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.185 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.185 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.185 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.186 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.186 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.186 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.186 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.186 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.186 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.186 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.187 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.187 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.187 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.187 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.187 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.187 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.187 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.187 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.188 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.188 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.188 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.188 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.188 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.188 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.188 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.188 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.189 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.189 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.189 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.189 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.189 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.189 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.189 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.189 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.190 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.190 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.190 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.190 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.190 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.190 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.190 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.191 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.191 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.191 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.191 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.191 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.191 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.191 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.192 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.192 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.192 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.192 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.192 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.192 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.192 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.192 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.193 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.193 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.193 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.193 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.193 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.193 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.193 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.194 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.194 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.194 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.194 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.194 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.194 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.194 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.194 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.195 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.195 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.195 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.195 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.195 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.195 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.195 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.195 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.196 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.196 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.196 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.196 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.196 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.196 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.196 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.196 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.197 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.197 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.197 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.197 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.197 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.197 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.197 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.197 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.198 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.198 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84623 signal_ha root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.198 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.198 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.198 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.198 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.198 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.199 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.199 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.199 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.199 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.199 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.199 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.199 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.200 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.200 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.200 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.200 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.200 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.200 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.200 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.201 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.201 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.201 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.201 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.201 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.201 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.201 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.201 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.202 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.202 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.202 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.202 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.202 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.202 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.202 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.202 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.203 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.203 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.203 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.203 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.203 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.203 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.203 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.203 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.204 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.204 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.204 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.204 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.204 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.204 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.204 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.205 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.205 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.205 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.205 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.205 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.205 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.205 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.205 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.206 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.206 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.206 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.206 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.206 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.206 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.206 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.206 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.207 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.207 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.207 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.207 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.207 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.207 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.207 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.208 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.208 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.208 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.208 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.208 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.208 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.208 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.209 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.209 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.209 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.209 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.209 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.209 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.209 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.209 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.210 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.210 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.210 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.210 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.210 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.210 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.210 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.210 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.211 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.211 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.211 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.211 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.211 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.211 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.211 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84624 rocksdb:l root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.211 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.212 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.212 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.212 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.212 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.212 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.212 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.212 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.213 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.213 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.213 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.213 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.213 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.213 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.213 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.213 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.214 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.214 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.214 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.214 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.214 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.214 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.214 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.214 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.215 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.215 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.215 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.215 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.215 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.215 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.216 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.216 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.216 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.216 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.216 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.216 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.216 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.216 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.217 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.217 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.217 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.217 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.217 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.217 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.217 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.217 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.218 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.218 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.218 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.218 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.218 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.218 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.218 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.218 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.219 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.219 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.219 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.219 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.219 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.219 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.219 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.219 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.220 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.220 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.220 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.220 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.220 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.220 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.220 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.221 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.221 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.221 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.221 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.221 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.221 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.221 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.221 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.222 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.222 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.222 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.222 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.222 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.222 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.222 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.222 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.223 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.223 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.223 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.223 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.223 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.223 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.223 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.224 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.224 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.224 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.224 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.224 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.224 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.224 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.224 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.225 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.225 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.225 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84625 rocksdb:h root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.225 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.225 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.225 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.225 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.226 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.226 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.226 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.226 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.226 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.226 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.226 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.226 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.227 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.227 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.227 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.227 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.227 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.227 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.227 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.227 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.228 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.228 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.228 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.228 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.228 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.228 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.228 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.228 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.230 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.230 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.230 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.230 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.230 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.230 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.230 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.230 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.231 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.231 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.231 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.231 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.231 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.231 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.231 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.232 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.232 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.232 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.232 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.232 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.232 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.233 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.233 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.233 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.233 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.233 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.233 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.233 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.233 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.234 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.234 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.234 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.234 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.234 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.234 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.234 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.234 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.235 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.235 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.235 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.235 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.235 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.235 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.235 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.235 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.236 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.236 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.236 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.236 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.236 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.236 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.236 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.237 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.237 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.237 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.237 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.237 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.237 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.237 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.237 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.238 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.238 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.238 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.238 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.238 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.238 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.238 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84641 ceph-mon root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.238 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.239 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.239 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.239 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.239 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.239 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.239 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.239 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.239 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.240 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.240 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.240 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.240 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.240 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.240 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.241 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.241 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.241 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.241 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.241 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.241 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.241 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.241 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.242 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.242 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.242 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.242 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.242 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.242 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.242 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.242 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.243 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.243 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.243 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.243 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.243 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.243 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.243 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.243 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.244 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.244 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.244 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.244 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.244 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.244 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.244 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.245 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.245 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.245 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.245 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.245 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.245 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.245 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.245 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.246 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.246 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.246 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.246 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.246 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.246 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.246 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.246 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.247 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.247 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.247 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.247 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.247 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.247 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.247 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.247 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.248 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.248 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.248 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.248 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.248 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.248 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.249 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.249 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.249 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.249 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.249 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.249 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.249 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.249 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.250 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.250 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.250 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.250 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.250 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.250 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.250 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.250 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.251 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.251 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.251 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.251 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.251 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.251 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.251 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.251 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.252 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.252 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.252 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84642 fn_monsto root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.252 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.252 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.252 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.252 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.252 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.253 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.253 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.253 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.253 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.253 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.253 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.253 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.253 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.254 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.254 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.254 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.254 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.254 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.254 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.254 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.254 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.255 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.255 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.255 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.255 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.255 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.255 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.255 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.256 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.256 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.256 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.256 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.256 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.256 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.256 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.257 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.257 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.257 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.257 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.257 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.257 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.257 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.257 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.258 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.258 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.258 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.258 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.258 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.258 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.258 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.258 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.259 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.259 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.259 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.259 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.259 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.259 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.259 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.259 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.260 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.260 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.260 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.260 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.260 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.260 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.260 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.261 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.261 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.261 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.261 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.261 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.261 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.261 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.261 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.262 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.262 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.262 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.262 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.262 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.262 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.262 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.262 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.263 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.263 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.263 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.263 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.263 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.263 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.263 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.263 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.264 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.264 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.264 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.264 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.264 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.264 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.264 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.265 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.265 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.265 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.265 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.265 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.265 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84643 msgr-work root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.265 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.266 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.266 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.266 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.266 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.266 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.266 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.266 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.266 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.267 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.267 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.267 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.267 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.267 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.267 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.267 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.267 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.268 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.268 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.268 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.268 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.268 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.268 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.268 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.268 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.271 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.271 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.271 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.271 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.271 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.271 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.271 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.271 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.272 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.272 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.272 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.272 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.272 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.272 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.272 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.273 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.273 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.273 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.273 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.273 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.273 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.273 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.274 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.274 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.274 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.274 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.274 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.274 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.274 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.274 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.275 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.275 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.275 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.275 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.275 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.275 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.275 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.275 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.276 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.276 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.276 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.276 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.276 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.276 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.276 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.276 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.277 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.277 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.277 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.277 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.277 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.277 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.277 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.277 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.278 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.278 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.278 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.278 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.278 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.278 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.278 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.279 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.279 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84644 msgr-work root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.279 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.279 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.279 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.279 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.279 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.279 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.280 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.280 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.280 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.280 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.280 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.280 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.280 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.280 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.281 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.281 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.281 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.281 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.281 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.281 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.281 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.282 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.282 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.282 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.282 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.282 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.282 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.282 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.283 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.283 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.283 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.283 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.283 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.283 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.283 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.283 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.284 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.284 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.284 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.284 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.284 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.284 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.284 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.284 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.285 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.285 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.285 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.285 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.285 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.285 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.285 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.285 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.291 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.291 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.291 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.291 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.291 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.291 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.291 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.291 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.292 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.292 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.292 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.292 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.292 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84645 msgr-work root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.292 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.292 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.293 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.293 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.293 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.293 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.293 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.293 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.293 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.293 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.294 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.294 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.294 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.294 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.294 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.294 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.294 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.294 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.295 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.295 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.295 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.295 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.295 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.295 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.295 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.295 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.296 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.296 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.296 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.296 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.296 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.296 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.296 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.297 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.297 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.297 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.297 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.297 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.297 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.297 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.298 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.298 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.298 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.298 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.298 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.298 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.298 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.298 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.299 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.299 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.299 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.299 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.299 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.299 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.299 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.299 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.300 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.300 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.300 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.300 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.300 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.300 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.300 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.301 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.301 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.301 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.301 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.301 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.301 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.301 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.301 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.302 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.302 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.302 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.302 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.302 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.302 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.302 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.302 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.303 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.303 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.303 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.303 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.303 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.303 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.303 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.303 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.304 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.304 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.304 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.304 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.305 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.305 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.305 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.305 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.305 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.305 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.305 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.305 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.306 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.306 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.306 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.306 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84646 OpHistory root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.306 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.307 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.307 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.307 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.307 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.307 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.307 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.307 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.308 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.308 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.308 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.308 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.308 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.308 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.308 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.308 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.309 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.309 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.309 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.309 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.309 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.309 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.309 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.310 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.310 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.310 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.310 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.310 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.310 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.310 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.310 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.311 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.311 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.311 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.311 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.311 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.311 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.311 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.311 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.312 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.312 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.312 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.312 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.312 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.312 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.312 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.312 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.313 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.313 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.313 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.313 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.313 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.313 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.313 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.314 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.314 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.314 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.314 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.314 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.314 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.315 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.315 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.315 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.315 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.315 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.315 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.315 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.315 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.316 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.316 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.316 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.316 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.316 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.316 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.316 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.317 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.317 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.317 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.317 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.317 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.317 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.317 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.317 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.318 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.318 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.318 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.318 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.318 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.318 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.318 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.318 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.319 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.319 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.319 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.319 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.319 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.319 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.319 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.319 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.320 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.320 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.320 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.320 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84647 fin root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.320 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.320 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.320 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.321 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.321 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.321 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.321 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.321 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.321 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.321 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.321 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.322 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.322 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.322 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.322 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.322 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.322 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.322 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.322 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.323 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.323 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.323 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.323 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.323 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.323 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.324 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.324 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.324 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.324 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.324 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.324 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.324 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.325 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.325 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.325 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.325 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.325 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.325 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.325 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.325 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.326 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.326 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.326 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.326 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.326 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.326 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.326 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.326 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.327 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.327 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.327 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.327 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.327 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.327 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.327 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.327 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.328 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.328 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.328 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.328 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.328 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.328 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.328 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.328 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.329 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.329 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.329 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.329 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.329 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.329 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.329 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.330 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.330 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.330 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.330 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.330 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.330 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.330 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.330 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.331 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.331 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.331 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.331 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.331 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.331 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.332 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.332 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.332 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.332 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.332 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.332 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.332 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.332 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.334 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.334 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84648 safe_time root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.334 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.334 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.334 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.334 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.334 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.335 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.335 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.335 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.335 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.335 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.335 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.335 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.335 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.336 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.336 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.336 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.336 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.336 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.336 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.336 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.336 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.337 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.337 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.337 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.337 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.337 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.337 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.337 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.337 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.338 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.338 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.338 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.338 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.338 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.338 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.338 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.339 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.339 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.339 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.339 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.339 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.339 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.339 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.340 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.340 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.340 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.340 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.340 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.340 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.340 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.341 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.341 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.341 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.341 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.341 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.341 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.341 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.341 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.342 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.342 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.342 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.342 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.342 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.342 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.342 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.342 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.343 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.343 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.343 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.343 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.343 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.343 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.343 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.344 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.344 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.344 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.344 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.344 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.344 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.344 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.344 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.345 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.345 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.345 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.345 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.345 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.345 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.345 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.345 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.346 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.346 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.346 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.346 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.346 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.346 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.346 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.346 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.347 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.347 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.347 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.347 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.347 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.347 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84649 cpu_tp root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.347 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.348 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.348 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.348 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.348 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.348 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.348 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.348 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.349 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.349 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.349 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.349 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.349 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.349 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.349 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.350 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.350 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.350 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.350 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.350 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.350 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.350 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.350 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.351 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.351 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.351 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.351 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.351 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.351 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.351 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.351 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.352 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.352 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.352 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.352 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.352 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.352 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.352 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.353 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.353 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.353 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.353 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.353 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.353 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.353 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.353 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.354 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.354 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.354 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.354 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.354 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.354 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.355 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.355 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.355 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.355 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.355 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.355 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.355 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.355 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.356 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.356 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.356 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.356 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.358 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.358 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.358 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.358 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.358 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.358 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.358 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.359 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.359 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.359 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.359 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.359 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.359 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.359 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.360 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.360 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.360 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.360 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.360 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.360 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.360 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.361 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.361 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.361 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.361 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.361 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.361 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.361 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.361 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.362 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.362 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84650 cpu_tp root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.362 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.362 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.362 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.362 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.362 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.363 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.363 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.363 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.363 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.363 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.363 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.363 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.363 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.364 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.364 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.364 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.364 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.364 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.364 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.364 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.365 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.365 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.365 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.365 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.365 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.365 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.366 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.366 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.366 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.366 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.366 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.366 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.366 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.366 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.367 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.367 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.367 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.367 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.367 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.367 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.367 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.368 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.368 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.368 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.368 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.368 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.368 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.368 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.368 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.369 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.369 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.369 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.369 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.369 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.369 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.369 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.370 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.370 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.370 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.370 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.370 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.370 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.370 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.370 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.371 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.371 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.371 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.371 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.371 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.371 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.371 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.371 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.372 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.372 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.372 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.372 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.372 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.372 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.372 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.373 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.373 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.373 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.373 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.373 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.373 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.373 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.374 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.374 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.374 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.374 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.374 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.374 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.374 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.375 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.375 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.375 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.375 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.375 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.375 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.375 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.376 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.376 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.376 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84651 cpu_tp root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.376 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.376 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.376 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.376 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.376 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.377 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.377 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.377 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.377 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.377 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.377 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.377 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.378 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.378 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.378 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.378 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.378 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.378 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.378 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.378 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.379 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.379 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.379 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.379 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.379 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.379 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.379 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.379 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.380 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.380 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.380 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.380 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.380 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.380 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.380 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.381 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.381 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.381 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.381 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.381 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.381 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.381 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.381 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.382 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.382 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.382 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.382 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.382 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.382 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.383 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.383 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.383 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.383 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.383 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.383 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.383 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.384 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.384 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.384 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.384 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.384 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.384 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.384 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.384 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.385 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.385 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.385 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.385 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.385 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.385 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.385 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.386 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.386 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.386 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.386 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.386 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.386 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.386 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.386 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.387 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.387 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.387 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.387 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.387 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.387 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.387 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.387 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.388 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.388 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.388 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.388 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.388 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.388 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.388 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.389 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.389 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.389 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.389 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.389 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.389 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.389 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.389 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.390 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84652 cpu_tp root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.390 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.390 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.390 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.390 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.390 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.391 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.391 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.391 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.391 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.391 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.391 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.391 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.392 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.392 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.392 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.392 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.392 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.392 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.392 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.393 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.393 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.393 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.393 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.393 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.393 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.393 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.393 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.394 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.394 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.394 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.394 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.394 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.394 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.394 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.394 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.395 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.395 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.395 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.395 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.395 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.395 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.395 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.396 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.396 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.396 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.396 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.396 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.396 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.396 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.396 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.397 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.397 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.397 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.397 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.397 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.397 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.397 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.397 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.398 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.398 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.398 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.398 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.398 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.398 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.398 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.399 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.399 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.399 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.399 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.399 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.399 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.400 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.400 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.400 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.400 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.400 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.400 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.400 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.401 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.401 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.401 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.401 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.401 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.401 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.401 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.401 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.402 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.402 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.402 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.402 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.402 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.402 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.402 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.402 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.403 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.403 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.403 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.403 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.403 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.403 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.403 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.404 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.404 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84653 ms_dispat root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.404 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.404 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.404 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.404 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.404 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.404 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.405 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.405 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.405 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.405 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.405 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.405 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.405 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.406 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.406 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.406 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.406 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.406 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.406 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.406 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.406 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.407 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.407 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.407 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.407 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.407 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.407 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.408 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.408 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.408 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.408 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.408 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.408 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.408 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.409 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.409 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.409 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.409 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.409 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.409 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.409 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.409 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.410 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.410 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.410 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.410 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.410 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.410 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.410 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.410 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.411 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.411 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.411 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.411 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.411 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.411 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.411 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.412 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.412 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.412 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.412 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.412 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.412 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.412 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.412 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.413 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.413 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.413 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.413 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.413 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.413 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.413 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.415 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.415 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.415 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.415 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.415 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.415 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.415 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.416 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.416 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.416 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.416 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.416 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.416 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.417 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.417 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.417 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.417 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.417 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.417 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.417 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.417 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.418 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.418 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84654 ms_local root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.418 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.418 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.418 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.418 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.418 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.419 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.419 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.419 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.419 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.419 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.419 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.419 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.419 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.420 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.420 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.420 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.420 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.420 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.420 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.420 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.421 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.421 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.421 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.421 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.421 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.421 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.421 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.421 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.422 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.422 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.422 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.422 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.422 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.422 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.422 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.423 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.423 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.423 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.423 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.423 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.423 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.423 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.423 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.424 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.424 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.424 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.424 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.424 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.424 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.425 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.425 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.425 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.425 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.425 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.425 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.425 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.426 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.426 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.426 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.426 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.426 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.426 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.426 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.427 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.427 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.427 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.427 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.427 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.427 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.427 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.427 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.428 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.428 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.428 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.428 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.428 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.428 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.428 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.429 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.429 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.429 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.429 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.429 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.429 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.429 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.429 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.430 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.430 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.430 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.430 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.430 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.430 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.430 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.430 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.431 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.431 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.431 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.431 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.431 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.431 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.431 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.432 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.432 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84655 safe_time root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.432 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.432 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.432 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.432 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.432 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.432 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.433 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.433 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.433 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.433 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.433 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.433 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.434 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.434 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.434 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.434 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.434 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.434 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.434 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.435 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.435 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.435 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.435 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.435 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.435 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.435 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.435 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.436 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.436 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.436 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.436 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.436 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.436 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.436 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.437 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.437 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.437 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.437 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.437 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.437 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.437 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.437 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.438 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.438 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.438 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.438 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.438 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.438 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.438 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.439 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.439 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.439 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.439 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.439 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.439 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.439 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.439 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.440 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.440 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.440 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.440 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.440 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.440 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.440 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.441 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.441 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.441 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.441 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.441 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.441 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.442 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.442 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.442 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.442 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.442 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.442 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.442 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.443 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.443 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.443 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.443 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.443 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.443 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.443 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.443 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.444 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.444 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.444 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.444 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.444 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.444 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.444 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.445 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.445 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.445 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.445 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.445 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.445 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.445 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.445 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.446 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.446 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.446 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84656 ms_dispat root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.446 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.446 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.446 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.446 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.447 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.447 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.447 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.447 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.447 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.447 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.447 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.447 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.448 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.448 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.448 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.448 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.448 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.448 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.448 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.449 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.449 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.449 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.449 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.449 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.449 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.449 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.449 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.450 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.450 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.450 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.450 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.450 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.450 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.451 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.451 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.451 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.451 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.451 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.451 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.451 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.452 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.452 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.452 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.452 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.452 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.452 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.452 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.452 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.453 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.453 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.453 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.453 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.453 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.453 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.453 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.454 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.454 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.454 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.454 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.454 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.454 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.454 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.454 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.455 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.455 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.455 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.455 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.455 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.455 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.455 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.455 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.456 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.456 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.456 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.456 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.456 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.456 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.456 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.457 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.457 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.457 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.457 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.457 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.457 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.457 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.457 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.458 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.458 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.458 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.458 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.458 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.458 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.458 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.459 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.459 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.459 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.459 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.459 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.459 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.460 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.460 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.460 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.460 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 84657 ms_local root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.460 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.460 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.460 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.460 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.461 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.461 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.461 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.461 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 9711464 263844 /usr/lib64/ceph/libceph-common.so.2 2023-07-29T14:21:37.461 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.461 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.461 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.462 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.462 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.462 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.462 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.462 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.462 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.462 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.462 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.463 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.463 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.463 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.463 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.463 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.463 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.463 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.464 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.464 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.464 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.464 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.464 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.464 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.464 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.464 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.465 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.465 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.465 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.465 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.465 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.465 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.465 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.465 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.466 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.466 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.466 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.466 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.466 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.466 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.466 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.467 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.467 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.467 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.467 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.467 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.467 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.468 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root mem REG 8,1 48 56840 /tmp/vgdb-pipe-shared-mem-vgdb-84610-by-root-on-??? 2023-07-29T14:21:37.468 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.468 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 1w FIFO 0,13 0t0 178954 pipe 2023-07-29T14:21:37.468 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 2w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.468 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 3w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.468 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 4r FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.468 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 5w FIFO 0,13 0t0 178982 pipe 2023-07-29T14:21:37.468 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 6w REG 8,1 161717065 263436 /var/log/ceph/ceph-mon.b.log 2023-07-29T14:21:37.469 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 7uW REG 0,24 6 2081 /run/ceph/ceph-mon.b.pid 2023-07-29T14:21:37.469 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 8r FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.469 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 9w FIFO 0,13 0t0 178985 pipe 2023-07-29T14:21:37.469 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 10u unix 0xffff89b9d821b740 0t0 178986 /var/run/ceph/ceph-mon.b.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.469 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 11r FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.469 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 12w FIFO 0,13 0t0 178988 pipe 2023-07-29T14:21:37.469 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 13r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.470 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 14uW REG 253,4 0 137 /var/lib/ceph/mon/ceph-b/store.db/LOCK 2023-07-29T14:21:37.470 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 15r DIR 253,4 4096 136 /var/lib/ceph/mon/ceph-b/store.db 2023-07-29T14:21:37.470 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 16w REG 253,4 120856 140 /var/lib/ceph/mon/ceph-b/store.db/000838.log 2023-07-29T14:21:37.470 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 17u IPv4 201288 0t0 TCP smithi045.front.sepia.ceph.com:3300->smithi045.front.sepia.ceph.com:33696 (ESTABLISHED) 2023-07-29T14:21:37.470 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 18w REG 253,4 70802 145 /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2023-07-29T14:21:37.470 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 19u a_inode 0,14 0 2079 [eventpoll:20,28,29] 2023-07-29T14:21:37.470 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 20r FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.470 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 21w FIFO 0,13 0t0 184404 pipe 2023-07-29T14:21:37.471 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 22u a_inode 0,14 0 2079 [eventpoll:23] 2023-07-29T14:21:37.471 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 23r FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.471 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 24w FIFO 0,13 0t0 184405 pipe 2023-07-29T14:21:37.471 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 25u a_inode 0,14 0 2079 [eventpoll:17,26] 2023-07-29T14:21:37.471 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 26r FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.471 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 27w FIFO 0,13 0t0 184406 pipe 2023-07-29T14:21:37.471 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 28u IPv4 178990 0t0 TCP smithi045.front.sepia.ceph.com:3300 (LISTEN) 2023-07-29T14:21:37.472 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 29u IPv4 178991 0t0 TCP smithi045.front.sepia.ceph.com:smc-https (LISTEN) 2023-07-29T14:21:37.472 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 30r FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.472 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 31w FIFO 0,13 0t0 183388 pipe 2023-07-29T14:21:37.472 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 32r FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.472 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 33w FIFO 0,13 0t0 183389 pipe 2023-07-29T14:21:37.472 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 34r FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.472 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 35w FIFO 0,13 0t0 183390 pipe 2023-07-29T14:21:37.472 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 36r REG 253,4 7028615 139 /var/lib/ceph/mon/ceph-b/store.db/000840.sst 2023-07-29T14:21:37.473 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 37w REG 8,1 685111 263438 /var/log/ceph/ceph.audit.log 2023-07-29T14:21:37.473 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root 38w REG 8,1 611534 263426 /var/log/ceph/ceph.log 2023-07-29T14:21:37.473 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root *524r REG 8,1 12239904 58269 /usr/bin/ceph-mon 2023-07-29T14:21:37.473 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root *525u REG 8,1 32 58779 /tmp/valgrind_proc_84610_cmdline_fc83ffdf (deleted) 2023-07-29T14:21:37.473 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root *526u REG 8,1 336 58781 /tmp/valgrind_proc_84610_auxv_fc83ffdf (deleted) 2023-07-29T14:21:37.473 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root *527w FIFO 0,13 0t0 178955 pipe 2023-07-29T14:21:37.473 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root *528w REG 8,1 1303 395758 /var/log/ceph/valgrind/mon.b.log 2023-07-29T14:21:37.474 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root *529r FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.474 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root *530w FIFO 0,13 0t0 178981 pipe 2023-07-29T14:21:37.474 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-mon 84610 88632 rstore_co root *531r FIFO 8,1 0t0 58786 /tmp/vgdb-pipe-from-vgdb-to-84610-by-root-on-??? 2023-07-29T14:21:37.474 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.474 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.474 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root txt REG 8,1 185296 11174 /usr/bin/sudo 2023-07-29T14:21:37.474 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 357408 9535 /usr/lib64/libnss_systemd.so.2 2023-07-29T14:21:37.474 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:37.475 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.475 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:37.475 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.475 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 19600 10382 /usr/lib64/libpam_misc.so.0.82.1 2023-07-29T14:21:37.475 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 19400 26916 /usr/lib64/security/pam_succeed_if.so 2023-07-29T14:21:37.475 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 514272 27100 /usr/lib64/security/pam_systemd.so 2023-07-29T14:21:37.475 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 27536 26895 /usr/lib64/security/pam_limits.so 2023-07-29T14:21:37.476 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 48392 9642 /usr/lib64/libcrack.so.2.9.0 2023-07-29T14:21:37.476 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 32608 10375 /usr/lib64/libpwquality.so.1.0.2 2023-07-29T14:21:37.476 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 15352 26893 /usr/lib64/security/pam_keyinit.so 2023-07-29T14:21:37.476 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 15840 26832 /usr/lib64/security/pam_pwquality.so 2023-07-29T14:21:37.476 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 14928 26881 /usr/lib64/security/pam_deny.so 2023-07-29T14:21:37.476 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 56712 26921 /usr/lib64/security/pam_unix.so 2023-07-29T14:21:37.477 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.477 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.477 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.477 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.477 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.477 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.477 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.477 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.478 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.478 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.478 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.478 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:37.478 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.478 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.478 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.478 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.479 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.479 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:37.479 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.479 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 15184 26885 /usr/lib64/security/pam_faildelay.so 2023-07-29T14:21:37.479 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:37.479 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 19464 26883 /usr/lib64/security/pam_env.so 2023-07-29T14:21:37.479 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 472584 48691 /usr/libexec/sudo/sudoers.so 2023-07-29T14:21:37.480 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:37.480 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:37.480 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:37.480 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:37.480 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:37.480 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:37.480 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:37.480 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:37.481 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:37.481 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:37.481 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.481 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:37.481 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 119832 48687 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-07-29T14:21:37.481 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.481 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:37.482 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:37.482 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.482 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:37.482 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root 0r FIFO 0,13 0t0 185403 pipe 2023-07-29T14:21:37.482 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.482 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.482 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root 3r FIFO 0,13 0t0 185421 pipe 2023-07-29T14:21:37.482 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root 4w FIFO 0,13 0t0 185421 pipe 2023-07-29T14:21:37.483 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root 5u netlink 0t0 185428 AUDIT 2023-07-29T14:21:37.483 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 85129 root 6u unix 0xffff89b9c39d0000 0t0 185429 type=DGRAM (CONNECTED) 2023-07-29T14:21:37.483 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.483 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.483 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root txt REG 8,1 15448 9913 /usr/bin/python3.9 2023-07-29T14:21:37.483 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 178728 9108 /usr/lib64/liblzma.so.5.2.5 2023-07-29T14:21:37.483 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 76792 9115 /usr/lib64/libbz2.so.1.0.8 2023-07-29T14:21:37.483 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.484 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 71616 32371 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.484 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 23800 32354 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.484 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 20000 32370 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.484 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 56416 32361 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.484 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.485 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 37592 32378 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.485 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 23832 32369 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.485 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 28064 32345 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.485 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 46560 32348 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.485 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 29232 32326 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.485 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 41752 32384 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-07-29T14:21:37.485 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:37.485 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 3474032 9915 /usr/lib64/libpython3.9.so.1.0 2023-07-29T14:21:37.486 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.486 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:37.486 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.486 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root 0r FIFO 0,13 0t0 185403 pipe 2023-07-29T14:21:37.486 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.486 INFO:teuthology.orchestra.run.smithi045.stdout:daemon-he 85177 root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.486 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.486 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85271 log root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85277 msgr-work root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85280 msgr-work root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85281 msgr-work root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85299 service root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85300 admin_soc root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85320 signal_ha root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85321 io_contex root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85323 io_contex root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85340 OpHistory root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85341 ceph_time root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85354 memcheck- root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.768 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.768 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.768 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.768 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.768 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.768 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.768 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85367 memcheck- root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.801 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.801 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.801 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.801 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.801 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.801 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.801 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.802 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.802 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.802 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.802 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.802 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.802 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.802 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.802 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.804 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.804 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.804 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.804 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.804 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.804 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85376 memcheck- root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85388 memcheck- root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.830 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.830 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.830 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.830 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.830 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.830 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.830 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.831 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.831 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.831 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.831 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.831 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.831 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.831 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.833 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.833 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.833 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.833 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.833 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.833 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.833 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.847 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.847 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.847 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.847 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.847 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.847 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.847 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.848 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.848 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.848 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.848 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.848 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.848 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.848 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85397 memcheck- root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.850 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.850 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.850 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.850 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.850 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.850 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.850 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.852 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.852 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.852 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.852 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.852 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.852 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.852 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.852 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.853 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.853 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.853 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.853 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.853 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.853 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.853 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.856 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.856 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.856 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.856 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.856 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.856 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.856 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.859 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.859 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.859 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.859 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.859 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.859 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.859 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.862 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.862 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.862 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.862 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.862 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.862 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.862 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85400 memcheck- root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.885 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.885 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.885 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.885 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.885 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.885 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.885 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.885 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.886 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.886 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.886 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.886 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.886 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.886 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.886 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.887 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.887 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.887 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.887 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.887 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.887 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.887 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.890 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.890 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.890 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.890 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.890 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.890 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.890 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.890 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85411 memcheck- root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.904 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.904 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.904 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.904 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.904 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.904 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.906 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.906 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.906 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.906 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.906 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.906 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.906 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.908 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.908 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.908 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.908 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.908 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.908 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.908 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.912 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.912 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.912 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.912 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.912 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.912 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.912 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85424 memcheck- root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.919 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.919 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.919 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.919 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.919 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.919 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.919 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.920 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.920 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.920 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.920 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.920 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.920 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.920 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.933 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.933 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.933 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.933 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.933 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.933 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.933 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85428 memcheck- root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.935 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.935 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.935 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.935 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.935 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.935 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.935 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.936 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.936 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.936 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.936 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.936 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.936 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.936 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.936 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.937 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.937 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.937 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.937 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.937 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.937 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.937 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.953 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.953 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.953 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.953 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.953 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.953 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.955 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.955 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.955 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.955 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.955 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.955 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.955 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.956 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.956 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85431 safe_time root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.956 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.956 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.956 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.956 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.956 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.956 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.958 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.958 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.958 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.958 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.958 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.958 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.958 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:37.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:37.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:37.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:37.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:37.960 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:37.960 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:37.960 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:37.960 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:37.961 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:37.961 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:37.961 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:37.961 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:37.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:37.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:37.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:37.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:37.963 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:37.963 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:37.963 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:37.963 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:37.964 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:37.964 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:37.964 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:37.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:37.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:37.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:37.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:37.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:37.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:37.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:37.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:37.967 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:37.967 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:37.967 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:37.967 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:37.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:37.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:37.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:37.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:37.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:37.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:37.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:37.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:37.970 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:37.970 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:37.970 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:37.970 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:37.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:37.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:37.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:37.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:37.972 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:37.972 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:37.972 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:37.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:37.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:37.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:37.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:37.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:37.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:37.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:37.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:37.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:37.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:37.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.976 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.976 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:37.976 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:37.976 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:37.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:37.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:37.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:37.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:37.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:37.979 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:37.979 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.979 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:37.979 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:37.980 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:37.980 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:37.980 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:37.980 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:37.981 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:37.981 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:37.981 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:37.982 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:37.982 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.982 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:37.982 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:37.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:37.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:37.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:37.984 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.984 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.984 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.984 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.985 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.985 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.985 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.985 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.986 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.986 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.986 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.986 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.987 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.987 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.987 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.987 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.989 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.989 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.989 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.989 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:37.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:37.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:37.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.991 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:37.991 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.991 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:37.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:37.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:37.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:37.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:37.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85434 safe_time root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:37.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:37.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:37.994 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:37.994 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:37.994 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:37.994 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:37.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:37.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:37.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:37.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:37.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:37.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:37.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:37.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:37.997 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:37.997 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:37.997 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:37.998 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:37.998 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:37.998 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:37.998 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:37.999 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:37.999 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:37.999 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:37.999 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.002 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.002 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.002 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.002 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.004 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.004 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.004 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.004 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.006 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.006 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.006 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.006 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.008 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.008 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.008 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.008 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.010 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.010 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.010 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.012 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.012 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.012 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.012 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.015 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.015 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.015 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.016 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.016 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.016 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.016 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.018 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.018 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.018 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.018 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.020 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.020 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.020 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.020 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.021 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.021 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.021 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.021 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.022 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.022 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.022 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.022 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85435 safe_time root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85437 safe_time root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85439 fn_anonym root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85472 rocksdb:l root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.105 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.105 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.105 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.105 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.105 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.105 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.105 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.106 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.106 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.106 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.106 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.106 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.106 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.106 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.106 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.107 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.107 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.107 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.107 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.107 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.107 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.107 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85473 rocksdb:l root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85474 rocksdb:l root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 85475 rocksdb:h root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86906 bstore_ai root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.192 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.192 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.192 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.192 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.192 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.192 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.192 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 86907 bstore_di root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.213 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.213 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.213 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.213 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.213 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.213 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.213 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.214 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.214 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.214 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.214 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.214 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.214 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.214 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.214 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.215 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.215 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.215 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.215 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.215 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.215 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.215 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.216 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.216 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.216 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.216 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.216 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.216 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.216 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.216 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.217 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.217 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.217 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.217 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.217 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.217 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.217 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.218 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.218 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.218 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.218 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.218 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.218 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.218 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.218 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.219 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.219 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.219 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.219 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87525 bstore_ai root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.219 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.219 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.219 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.219 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.220 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.220 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.220 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.220 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.220 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.220 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.220 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.221 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.221 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.221 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.221 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.221 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.221 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.221 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.222 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.222 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.222 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.222 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.222 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.222 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.222 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.222 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.223 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.223 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.223 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.223 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.223 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.223 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.223 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.227 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.227 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.227 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.227 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.227 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.227 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.227 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.227 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.228 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.228 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.228 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.228 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.228 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.228 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.228 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.228 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.229 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.229 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.229 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.229 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.229 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.229 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.229 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.230 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.230 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.230 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.230 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.230 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.230 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.230 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.231 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.231 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.231 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.231 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.231 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.231 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.231 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.231 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.232 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.232 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.232 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.232 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.232 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.232 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.232 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.232 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.233 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.233 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.233 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.233 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.233 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.233 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.233 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.234 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.234 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.234 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.234 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.234 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.234 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.234 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.234 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.235 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.235 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.235 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.235 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.235 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.235 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.235 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.235 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.236 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.236 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.236 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.236 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.236 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.236 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.236 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.237 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.237 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.237 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.237 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.237 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.237 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.237 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.238 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.238 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.238 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.238 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.238 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.238 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.238 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.238 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.239 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.239 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.239 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.239 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.239 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.239 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.239 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.240 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.240 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.240 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.240 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.240 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 87526 bstore_di root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.240 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.240 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.240 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.241 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.241 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.241 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.241 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.241 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.241 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.241 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.242 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.242 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.242 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.242 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.242 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.242 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.242 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.242 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.243 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.243 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.243 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.243 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.243 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.243 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.243 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.243 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.244 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.244 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.244 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.244 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.244 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.244 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.244 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.245 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.245 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.245 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.245 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.245 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.245 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.246 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.246 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.246 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.246 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.246 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.246 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.246 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.247 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.247 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.247 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.247 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.247 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.247 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.247 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.248 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.248 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.248 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.248 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.248 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.248 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.248 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.248 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.249 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.249 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.249 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.249 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.249 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.249 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.249 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.250 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.250 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.250 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.250 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.250 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.250 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.250 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.250 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.251 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.251 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.251 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.251 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.251 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.251 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.251 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.251 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.252 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.252 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.252 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.252 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.252 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.252 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.252 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.253 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.253 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.253 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.253 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.253 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.253 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.253 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.254 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.254 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.254 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.254 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.254 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.254 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.254 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.254 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.255 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.255 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.255 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.255 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.255 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.255 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.255 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.256 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.256 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.256 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.256 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.256 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.256 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.256 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.256 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.257 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.257 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.257 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.257 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.257 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.257 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.257 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.257 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.258 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.258 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.258 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.258 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.258 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.258 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.258 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.259 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.259 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.259 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.259 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.259 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.259 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.259 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.259 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.261 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.261 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.261 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.261 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.261 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.261 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.261 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88351 memcheck- root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.262 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.262 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.262 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.262 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.262 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.262 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.262 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.264 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.264 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.264 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.264 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.264 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.264 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.264 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.265 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.265 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.265 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.265 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.265 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.265 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.265 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.265 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.266 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.266 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.266 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.266 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.266 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.266 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.266 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.266 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.267 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.267 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.267 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.267 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.267 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.267 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.267 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.268 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.268 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.268 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.268 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.268 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.268 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.268 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.268 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.269 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.269 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.269 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.269 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.269 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.269 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.269 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.269 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.270 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.270 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.270 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.270 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.270 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.270 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.271 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.271 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.271 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.271 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.271 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.271 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.271 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.271 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.272 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.272 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.272 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.272 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.272 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.272 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.272 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.272 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.273 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.273 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.273 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.273 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.273 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.273 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.273 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.274 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.274 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.274 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.274 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.274 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.274 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.274 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.274 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.275 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.275 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.275 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.275 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.275 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.275 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.275 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.275 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.276 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.276 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.276 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.276 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.276 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.276 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.276 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.277 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.277 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.277 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.277 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.277 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.277 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.277 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.277 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.278 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.278 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.278 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.278 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.278 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.278 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.278 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.279 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.279 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.279 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.279 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.279 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.279 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.279 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.279 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.280 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.280 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.280 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.280 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.280 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.280 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.280 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.281 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.281 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.281 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.281 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.281 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.281 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.281 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.281 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.282 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.282 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.282 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.282 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.282 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.282 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.282 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.282 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88363 cfin root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.283 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.283 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.283 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.283 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.283 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.283 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.283 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.284 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.284 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.284 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.284 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.284 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.284 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.284 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.284 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.285 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.285 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.285 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.285 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.285 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.285 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.285 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.285 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.286 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.286 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.286 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.286 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.286 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.286 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.287 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.287 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.287 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.287 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.287 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.287 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.287 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.287 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.288 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.288 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.288 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.288 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.288 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.288 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.288 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.289 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.289 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.289 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.289 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.289 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.289 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.289 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.289 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.290 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.290 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.290 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.290 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.290 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.290 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.290 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.290 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.291 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.291 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.291 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.291 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.291 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.291 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.291 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.292 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.292 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.292 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.292 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.292 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.292 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.292 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.292 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.293 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.293 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.293 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.293 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.293 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.293 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.293 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.293 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.294 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.294 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.294 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.294 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.294 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.294 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.294 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.295 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.295 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.295 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.295 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.295 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.295 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.295 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.296 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.296 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.296 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.296 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.296 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.296 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.296 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.296 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.297 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.297 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.297 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.297 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.297 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.297 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.297 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.298 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.298 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.298 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.298 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.298 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.298 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.298 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.298 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.299 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.299 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.299 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.299 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.299 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.299 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.299 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.299 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.300 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.300 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.300 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.300 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.300 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.300 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.300 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.300 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.301 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.301 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.301 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.301 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.301 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.301 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.301 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.302 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.302 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.302 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.302 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.302 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.302 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.302 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.303 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.303 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.303 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.303 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.303 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.303 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.303 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.303 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.304 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88364 bstore_kv root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.304 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.304 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.304 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.304 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.304 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.304 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.305 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.305 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.305 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.305 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.305 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.305 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.305 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.305 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.306 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.306 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.306 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.306 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.306 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.306 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.306 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.307 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.307 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.307 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.307 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.307 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.307 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.307 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.307 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.308 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.308 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.308 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.308 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.308 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.308 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.308 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.308 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.309 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.309 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.309 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.309 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.309 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.309 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.309 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.310 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.310 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.310 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.310 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.310 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.310 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.310 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.310 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.311 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.311 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.311 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.311 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.311 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.311 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.311 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.312 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.312 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.312 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.312 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.312 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.312 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.312 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.313 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.313 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.313 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.313 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.313 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.313 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.313 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.313 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.314 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.314 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.314 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.314 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.314 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.314 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.314 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.314 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.315 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.315 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.315 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.315 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.315 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.315 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.315 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.316 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.316 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.316 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.316 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.316 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.316 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.316 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.316 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.317 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.317 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.317 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.317 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.317 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.317 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.317 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.319 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.319 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.319 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.319 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.319 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.319 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.319 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.320 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.320 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.320 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.320 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.320 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.320 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.320 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.321 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.321 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.321 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.321 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.321 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.321 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.321 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.321 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.322 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.322 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.322 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.322 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.322 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.322 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.322 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.322 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.323 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.323 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.323 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.323 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.323 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.323 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.323 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.323 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.324 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.324 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.324 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.324 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.324 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.324 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.324 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.325 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.325 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88365 bstore_kv root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.325 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.325 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.325 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.325 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.325 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.325 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.326 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.326 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.326 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.326 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.326 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.326 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.326 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.327 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.327 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.327 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.327 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.327 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.327 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.327 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.328 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.328 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.328 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.328 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.328 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.328 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.328 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.328 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.329 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.329 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.329 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.329 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.329 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.329 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.329 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.329 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.330 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.330 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.330 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.330 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.330 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.330 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.330 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.331 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.331 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.331 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.331 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.331 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.331 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.331 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.331 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.332 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.332 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.332 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.332 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.332 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.332 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.332 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.333 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.333 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.333 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.333 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.333 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.333 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.333 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.333 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.334 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.334 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.334 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.334 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.334 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.334 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.334 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.334 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.335 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.335 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.335 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.335 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.335 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.335 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.335 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.336 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.336 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.336 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.336 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.336 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.336 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.336 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.337 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.337 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.337 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.337 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.337 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.337 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.337 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.337 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.338 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.338 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.338 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.338 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.338 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.338 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.338 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.338 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.339 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.339 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.339 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.339 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.339 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.339 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.339 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.340 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.340 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.340 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.340 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.340 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.340 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.340 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.340 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.341 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.341 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.341 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.341 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.341 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.341 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.341 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.342 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.342 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.342 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.342 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.342 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.342 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.342 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.342 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.343 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.343 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.343 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.343 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.343 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.343 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.343 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.344 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.344 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.344 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.344 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.344 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.344 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.344 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.345 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.345 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.345 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.345 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.345 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.345 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.345 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.345 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.346 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.346 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.346 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88367 bstore_me root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.346 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.346 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.346 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.346 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.346 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.347 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.347 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.347 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.347 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.347 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.347 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.347 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.348 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.348 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.348 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.348 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.348 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.348 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.348 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.348 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.349 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.349 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.349 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.349 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.349 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.349 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.349 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.349 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.350 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.350 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.350 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.350 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.350 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.350 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.350 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.351 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.351 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.351 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.351 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.351 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.351 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.351 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.351 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.352 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.352 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.352 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.352 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.352 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.352 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.352 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.353 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.353 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.353 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.353 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.353 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.353 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.353 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.354 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.354 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.354 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.354 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.354 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.354 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.354 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.355 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.355 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.355 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.355 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.355 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.355 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.355 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.355 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.356 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.356 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.356 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.356 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.356 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.356 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.356 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.356 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.357 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.357 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.357 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.357 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.357 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.357 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.357 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.358 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.358 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.358 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.358 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.358 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.358 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.358 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.358 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.359 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.359 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.359 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.359 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.359 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.359 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.359 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.360 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.360 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.360 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.360 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.360 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.360 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.360 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.360 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.361 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.361 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.361 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.361 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.361 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.361 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.361 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.362 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.362 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.362 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.362 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.362 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.362 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.362 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.362 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.363 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.363 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.363 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.363 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.363 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.363 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.363 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.363 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.364 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.364 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.364 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.364 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.364 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.364 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.364 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.365 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.365 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.365 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.365 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.365 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.365 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.365 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.365 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.366 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.366 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.366 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.366 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.366 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.366 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.366 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.366 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.367 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.367 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.367 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88444 ms_dispat root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.367 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.367 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.367 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.367 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.367 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.368 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.368 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.368 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.368 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.368 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.368 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.369 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.369 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.369 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.369 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.369 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.369 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.369 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.369 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.370 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.370 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.370 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.370 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.370 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.370 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.370 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.370 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.371 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.371 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.371 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.371 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.371 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.371 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.371 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.371 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.372 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.372 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.372 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.372 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.372 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.372 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.372 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.373 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.373 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.373 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.373 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.373 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.373 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.373 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.373 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.374 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.374 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.374 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.374 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.374 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.374 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.374 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.374 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.375 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.375 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.375 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.375 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.375 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.375 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.375 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.376 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.376 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.376 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.376 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.376 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.376 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.376 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.377 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.377 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.377 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.377 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.377 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.377 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.377 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.377 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.378 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.378 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.378 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.378 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.378 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.378 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.378 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.378 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.379 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.379 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.379 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.379 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.379 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.379 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.379 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.379 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.380 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.380 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.380 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.380 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.380 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.380 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.380 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.380 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.381 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.381 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.381 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.381 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.381 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.381 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.381 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.381 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.382 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.382 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.382 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.382 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.382 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.382 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.382 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.383 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.383 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.383 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.383 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.383 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.383 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.383 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.383 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.384 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.384 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.384 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.384 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.384 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.384 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.384 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.385 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.385 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.385 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.385 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.385 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.385 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.385 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88445 ms_local root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.395 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.395 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.395 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.395 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.395 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.395 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.395 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.396 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.396 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.396 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.396 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.396 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.396 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.396 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.396 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.397 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.397 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.397 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.397 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.397 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.397 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.397 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.397 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.398 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.398 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.398 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.398 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.398 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.398 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.398 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.399 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.399 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.399 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.399 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.399 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.399 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.399 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.399 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.400 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.400 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.400 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.400 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.400 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.400 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.400 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.401 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.401 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.401 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.401 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.401 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.401 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.401 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.401 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.402 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.402 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.402 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.402 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.402 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.402 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.402 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.402 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.403 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.403 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.403 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.403 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.403 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.403 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.403 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.404 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.404 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.404 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.404 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.404 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.404 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.404 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.404 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.405 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.405 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.405 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.405 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.405 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.405 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.405 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.405 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.406 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.406 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.406 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.406 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.406 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.406 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.406 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.407 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.407 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.407 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.407 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.407 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.407 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.407 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.407 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.408 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.408 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.408 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.408 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.408 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.408 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.408 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.409 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.409 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88446 safe_time root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.409 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.409 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.409 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.409 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.409 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.409 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.410 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.410 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.410 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.410 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.410 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.410 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.410 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.410 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.411 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.411 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.411 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.411 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.411 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.411 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.411 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.412 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.412 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.412 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.412 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.412 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.412 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.412 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.412 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.413 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.413 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.413 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.413 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.413 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.413 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.413 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.413 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.414 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.414 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.414 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.414 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.414 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.414 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.414 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.414 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.415 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.415 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.415 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.415 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.415 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.415 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.415 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.416 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.416 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.416 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.416 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.416 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.416 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.416 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.417 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.417 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.417 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.417 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.417 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.417 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.417 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.417 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.419 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.419 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.419 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.419 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.419 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.419 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.419 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.420 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.420 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.420 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.420 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.420 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.420 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.420 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.421 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.421 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.421 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.421 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.421 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.421 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.421 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.421 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.422 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.422 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.422 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.422 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.422 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.422 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.422 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.423 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.423 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.423 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.423 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.423 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.423 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.423 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.423 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.424 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.424 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.424 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.424 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.424 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.424 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.424 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.425 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.425 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.425 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.425 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.425 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.425 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.425 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.425 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.426 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.426 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.426 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.426 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.426 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.426 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.426 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.427 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.427 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.427 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.427 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.427 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.427 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.427 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.427 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.428 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.428 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.428 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.428 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.428 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.428 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.428 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.428 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.429 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.429 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.429 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.429 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.429 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.429 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.429 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.429 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.430 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88447 safe_time root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.430 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.430 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.430 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.430 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.430 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.430 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.430 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.431 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.431 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.431 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.431 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.431 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.431 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.431 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.432 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.432 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.432 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.432 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.432 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.432 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.432 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.433 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.433 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.433 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.433 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.433 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.433 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.433 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.433 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.434 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.434 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.434 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.434 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.434 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.434 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.434 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.434 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.435 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.435 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.435 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.435 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.435 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.435 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.435 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.436 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.436 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.436 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.436 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.436 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.436 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.436 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.436 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.437 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.437 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.437 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.437 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.437 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.437 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.437 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.437 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.438 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.438 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.438 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.438 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.438 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.438 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.438 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.439 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.439 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.439 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.439 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.439 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.439 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.439 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.439 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.440 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.440 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.440 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.440 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.440 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.440 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.440 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.441 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.441 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.441 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.441 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.441 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.441 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.441 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.442 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.442 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.442 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.442 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.442 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.442 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.442 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.442 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.443 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.443 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.443 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.443 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.443 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.443 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.443 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.444 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.444 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.444 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.444 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.444 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.444 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.444 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.444 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.445 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.445 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.445 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.445 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.445 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.445 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.445 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.445 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.446 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.446 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.446 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.446 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.446 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.446 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.446 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.447 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.447 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.447 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.447 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.447 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.447 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.447 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.448 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.448 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.448 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.448 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.448 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.448 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.448 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.449 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.449 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.449 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.449 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.449 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.449 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.449 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.450 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.450 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.450 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.450 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.450 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.450 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.450 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.451 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.451 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.451 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.451 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88448 ms_dispat root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.451 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.451 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.451 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.451 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.452 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.452 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.452 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.452 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.452 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.452 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.452 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.453 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.453 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.453 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.453 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.453 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.453 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.453 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.453 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.454 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.454 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.454 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.454 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.454 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.454 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.454 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.455 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.455 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.455 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.455 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.455 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.455 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.455 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.455 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.456 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.456 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.456 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.456 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.456 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.456 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.456 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.457 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.457 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.457 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.457 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.457 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.458 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.458 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.458 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.458 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.458 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.458 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.458 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.458 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.459 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.459 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.459 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.459 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.459 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.459 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.459 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.460 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.460 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.460 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.460 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.460 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.460 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.460 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.460 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.461 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.461 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.461 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.461 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.461 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.461 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.461 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.461 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.462 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.462 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.462 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.462 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.462 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.462 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.462 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.462 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.463 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.463 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.463 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.463 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.463 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.463 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.463 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.464 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.464 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.464 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.464 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.464 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.464 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.464 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.464 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.465 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.465 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.465 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.465 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.465 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.465 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.465 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.466 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.466 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.466 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.466 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.466 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.466 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.466 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.466 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.467 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.467 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.467 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.467 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.467 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.467 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.467 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.468 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.468 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.468 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.468 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.468 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.468 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.468 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.468 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.469 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.469 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.469 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.469 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.469 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.469 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.469 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.469 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.471 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.471 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.471 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.471 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.471 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.471 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.471 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.471 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.472 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.472 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.472 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.472 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.472 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.472 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88449 ms_local root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.472 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.472 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.473 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.473 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.473 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.473 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.473 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.473 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.473 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.474 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.474 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.474 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.474 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.474 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.474 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.474 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.475 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.475 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.475 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.475 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.475 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.475 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.475 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.475 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.476 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.476 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.476 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.476 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.476 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.476 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.476 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.476 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.477 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.477 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.477 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.477 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.477 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.477 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.477 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.478 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.478 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.478 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.478 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.478 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.478 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.478 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.478 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.479 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.479 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.479 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.479 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.479 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.479 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.479 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.479 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.480 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.480 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.480 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.480 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.480 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.480 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.480 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.480 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.481 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.481 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.481 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.481 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.481 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.481 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.481 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.482 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.482 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.482 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.482 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.482 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.482 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.482 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.482 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.483 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.483 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.483 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.483 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.483 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.483 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.483 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.484 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.484 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.484 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.484 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.484 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.484 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.484 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.484 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.485 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.485 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.485 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.485 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.485 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.485 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.485 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.486 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.486 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.486 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.486 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.486 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.486 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.486 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.486 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.487 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.488 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.489 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.490 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.491 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.492 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88450 ms_dispat root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.493 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.494 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.495 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.496 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.497 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.498 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.499 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.500 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.501 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.502 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.503 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.505 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.506 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.507 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.508 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.509 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.510 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.511 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.512 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.513 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.514 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88451 ms_local root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.515 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.516 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.522 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.523 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.524 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.525 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.526 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.527 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.528 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.529 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.530 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.531 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.532 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.533 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.534 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.535 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88452 ms_dispat root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.536 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.537 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.538 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.539 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.540 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.541 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.542 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.543 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.544 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.545 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.546 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.547 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.548 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.549 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.550 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.551 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.552 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.553 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.554 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.555 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.556 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88453 ms_local root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.557 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.558 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.559 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.560 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.561 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.562 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.563 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.564 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.565 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.566 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.567 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.568 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.569 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.570 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.571 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.572 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.573 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.574 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.575 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.576 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.577 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88454 ms_dispat root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.578 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.579 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.580 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.581 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.582 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.583 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.584 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.585 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.586 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.587 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.588 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.589 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.590 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.591 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.592 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.593 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.594 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.595 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.596 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.597 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.598 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88455 ms_local root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.599 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.600 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.601 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.602 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.603 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.604 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.605 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.606 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.607 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.608 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.609 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.610 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.611 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.612 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.613 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.614 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.615 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.616 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.617 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.618 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.619 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88456 ms_dispat root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.620 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.621 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.622 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.623 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.624 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.625 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.626 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.627 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.628 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.629 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.630 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.631 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.638 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.639 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.640 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88457 ms_local root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.641 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.642 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.643 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.644 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.645 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.646 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.647 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.648 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.649 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.650 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.651 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.652 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.653 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.654 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.655 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.656 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.657 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.658 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.659 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.660 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.661 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88458 ms_dispat root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.662 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.663 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.664 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.665 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.666 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.667 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.668 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.669 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.670 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.671 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.672 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.673 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.674 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.675 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.676 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.677 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.678 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.679 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.680 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.681 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.682 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88459 ms_local root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.683 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.684 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.685 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.686 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.687 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.688 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.689 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.690 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.691 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.692 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.693 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.694 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.695 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.696 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.697 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.698 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.699 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.700 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.701 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.702 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.703 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88460 fn_anonym root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.704 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.705 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.706 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.707 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.708 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.709 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.710 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.711 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.712 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.713 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.714 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.715 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.717 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.718 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.719 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.720 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.721 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.722 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.723 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.724 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88461 finisher root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.725 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.726 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.727 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.728 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.729 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.730 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.731 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.732 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.733 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.734 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.735 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.736 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.737 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.738 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.739 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.740 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.741 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.742 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.743 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.744 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.745 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88462 safe_time root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.746 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.747 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.748 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.749 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.750 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.751 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.752 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.753 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.754 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.755 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.756 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.757 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.758 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.759 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.760 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.761 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.762 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.763 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.764 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.765 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.766 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88463 safe_time root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.767 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.768 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.768 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.768 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.768 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.769 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.770 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.771 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.772 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.773 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.774 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.775 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.776 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.777 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.778 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.779 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.785 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.786 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.787 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.788 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.789 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88464 memcheck- root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.790 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.791 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.792 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.793 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.794 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.795 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.796 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.797 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.798 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.799 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.800 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.801 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.801 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.801 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.801 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.802 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.802 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.802 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.803 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.804 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.804 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.804 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.804 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.804 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.805 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.806 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.807 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.808 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.809 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.810 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.811 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.812 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.813 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88465 osd_srv_a root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.814 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.815 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.816 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.817 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.818 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.819 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.820 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.821 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.824 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.825 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.826 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.827 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.828 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.829 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.830 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.830 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.830 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.830 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.831 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.831 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.831 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.832 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.833 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.833 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.833 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.833 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.834 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.835 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.836 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.837 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.838 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88466 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.839 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.840 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.841 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.842 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.843 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.844 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.845 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.846 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.847 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.847 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.847 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.847 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.848 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.848 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.848 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.849 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.850 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.850 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.850 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.851 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.852 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.852 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.852 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.853 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.853 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.853 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.853 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.854 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.855 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.856 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.856 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.856 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.857 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.858 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.859 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.859 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.859 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.860 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.861 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.862 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.862 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.862 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.862 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.863 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.864 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.865 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.866 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.867 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88467 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.868 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.869 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.870 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.871 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.872 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.873 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.874 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.875 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.876 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.877 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.878 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.879 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.880 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.881 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.882 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.883 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.884 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.885 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.885 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.885 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.886 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.886 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.886 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.886 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.887 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.887 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.887 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.888 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.889 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.890 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.890 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.890 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.891 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.892 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.893 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.894 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88468 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.895 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.896 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.897 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.898 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.899 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.900 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.901 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.902 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.903 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.904 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.904 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.904 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.905 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.906 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.906 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.906 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.906 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.907 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.908 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.908 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.908 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.909 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.910 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.911 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.912 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.912 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.912 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.912 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.913 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.914 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.915 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.916 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.917 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.918 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.919 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.919 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.919 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.919 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.920 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.920 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.920 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.920 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.921 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.922 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.924 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88469 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.925 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.926 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.927 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.928 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.929 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.930 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.931 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.932 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.933 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.933 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.933 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.933 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.934 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.935 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.935 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.935 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.935 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.936 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.936 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.936 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.937 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.937 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.937 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.937 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.938 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.939 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.940 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.941 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.942 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.943 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.944 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.945 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.946 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.947 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.948 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88470 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.949 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.950 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.951 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.952 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.953 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.953 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.953 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.954 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.955 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.955 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.955 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.955 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.956 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.956 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.956 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.957 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.958 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.958 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.958 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.959 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.960 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.960 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.960 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.960 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.961 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.961 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.961 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.961 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.962 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.963 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.963 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.963 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.963 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.963 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.963 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.963 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.964 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.964 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.964 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.964 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.964 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.964 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.964 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.964 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.965 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.966 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.967 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.967 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.967 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.967 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.967 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.967 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.967 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.968 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.969 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.970 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.970 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.970 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.970 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.970 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.970 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.970 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.971 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.972 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.972 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.972 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.972 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.972 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.972 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.972 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.972 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.973 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.974 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88471 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.975 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.976 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.976 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.976 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.976 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.976 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.976 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.976 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.977 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:38.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:38.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:38.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:38.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:38.978 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:38.979 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:38.979 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:38.979 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:38.979 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:38.979 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:38.979 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:38.979 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:38.980 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:38.980 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:38.980 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:38.980 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:38.980 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:38.980 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:38.981 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:38.981 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:38.981 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:38.981 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:38.981 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:38.981 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:38.981 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:38.982 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:38.982 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:38.982 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:38.982 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:38.982 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:38.982 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:38.982 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:38.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:38.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:38.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:38.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:38.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:38.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:38.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:38.983 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:38.984 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:38.984 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:38.984 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:38.984 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:38.984 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:38.984 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:38.984 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:38.985 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:38.985 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:38.985 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:38.985 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:38.985 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:38.985 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:38.985 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:38.986 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:38.986 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:38.986 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:38.986 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:38.986 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:38.986 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:38.986 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:38.987 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:38.987 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:38.987 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:38.987 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:38.987 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:38.987 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:38.987 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:38.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:38.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:38.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:38.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:38.988 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.989 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:38.989 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:38.989 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.989 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:38.989 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:38.989 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.989 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:38.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:38.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:38.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:38.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:38.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:38.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:38.990 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:38.991 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:38.991 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:38.991 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:38.991 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:38.991 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:38.991 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.991 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:38.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:38.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:38.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:38.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:38.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.992 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.993 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.994 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.994 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.994 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.994 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.994 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.994 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.994 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:38.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:38.995 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:38.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:38.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:38.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:38.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:38.996 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88472 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:38.998 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:38.998 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:38.998 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:38.998 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:38.998 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:38.998 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:38.998 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:38.999 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:38.999 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:38.999 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:38.999 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:38.999 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:38.999 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:38.999 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:39.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:39.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:39.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:39.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:39.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:39.000 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:39.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:39.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:39.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:39.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:39.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:39.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.001 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.002 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.002 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.002 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:39.002 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:39.002 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:39.002 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:39.002 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:39.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:39.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:39.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:39.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:39.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.003 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:39.004 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.004 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:39.004 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:39.004 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:39.004 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:39.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:39.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:39.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:39.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:39.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:39.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:39.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:39.005 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:39.006 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:39.006 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:39.006 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:39.006 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:39.006 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:39.006 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:39.006 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:39.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:39.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:39.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:39.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:39.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:39.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:39.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:39.007 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:39.008 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:39.008 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:39.008 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:39.008 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:39.008 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:39.008 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:39.008 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:39.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:39.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:39.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:39.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.009 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.010 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:39.010 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.010 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.010 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:39.010 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.010 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.010 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:39.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:39.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:39.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:39.011 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:39.012 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:39.012 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:39.012 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:39.012 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:39.012 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:39.012 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:39.012 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:39.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:39.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:39.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:39.013 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.014 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.015 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.015 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.015 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.015 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.015 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.015 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.015 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.016 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.016 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.016 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.016 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.016 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.016 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:39.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.017 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.018 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:39.018 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:39.018 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.018 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.018 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.018 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.018 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88473 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:39.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:39.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:39.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:39.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:39.019 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:39.020 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:39.020 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:39.020 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:39.020 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:39.020 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:39.020 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:39.020 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.021 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:39.021 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:39.021 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:39.021 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.022 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.022 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:39.022 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:39.022 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:39.022 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:39.022 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:39.022 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:39.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:39.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:39.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:39.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:39.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:39.023 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:39.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:39.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:39.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:39.024 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:39.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:39.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:39.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:39.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:39.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:39.025 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:39.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:39.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:39.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:39.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:39.026 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:39.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:39.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:39.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:39.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:39.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:39.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:39.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:39.027 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:39.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:39.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:39.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:39.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:39.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:39.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:39.028 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:39.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:39.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:39.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:39.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:39.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:39.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:39.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:39.029 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:39.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:39.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:39.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:39.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:39.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:39.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:39.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:39.030 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:39.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:39.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.031 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:39.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:39.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:39.032 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:39.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:39.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:39.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:39.033 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:39.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:39.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:39.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:39.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:39.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:39.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:39.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.034 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:39.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:39.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:39.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.035 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.036 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.037 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.038 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:39.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:39.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:39.039 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88474 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:39.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:39.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.040 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:39.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:39.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:39.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:39.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:39.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:39.041 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:39.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:39.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:39.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:39.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:39.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:39.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:39.042 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:39.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:39.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:39.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:39.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:39.043 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:39.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:39.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:39.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:39.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:39.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:39.044 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:39.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:39.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:39.045 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:39.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:39.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:39.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:39.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:39.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:39.046 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:39.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:39.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:39.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:39.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:39.047 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:39.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:39.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:39.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:39.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:39.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:39.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:39.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:39.048 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:39.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:39.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:39.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:39.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:39.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:39.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:39.049 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:39.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:39.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:39.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:39.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:39.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:39.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:39.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:39.050 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:39.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:39.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:39.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:39.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:39.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:39.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:39.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:39.051 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:39.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:39.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:39.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.052 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:39.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:39.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.053 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:39.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:39.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:39.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:39.054 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:39.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:39.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:39.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:39.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:39.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:39.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:39.055 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:39.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:39.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:39.056 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:39.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.057 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.058 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.059 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:39.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.060 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:39.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:39.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.061 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88475 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:39.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:39.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:39.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:39.062 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:39.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:39.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:39.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:39.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:39.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:39.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:39.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:39.063 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:39.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:39.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:39.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:39.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:39.064 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:39.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:39.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:39.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:39.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:39.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:39.065 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:39.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:39.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:39.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:39.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.066 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:39.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:39.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:39.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:39.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:39.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:39.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.067 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:39.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:39.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:39.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:39.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:39.068 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:39.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:39.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:39.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:39.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:39.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:39.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:39.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:39.069 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:39.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:39.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:39.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:39.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:39.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:39.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:39.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:39.070 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:39.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:39.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:39.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:39.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:39.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:39.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:39.071 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:39.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:39.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:39.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:39.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:39.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:39.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:39.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:39.072 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:39.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:39.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:39.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:39.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:39.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.073 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:39.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:39.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.074 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:39.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:39.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:39.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.075 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:39.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:39.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:39.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:39.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:39.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:39.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:39.076 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:39.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:39.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:39.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:39.077 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:39.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.078 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.079 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.080 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:39.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.081 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:39.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:39.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.082 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88476 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:39.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:39.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:39.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:39.085 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:39.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:39.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:39.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:39.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:39.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:39.086 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:39.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:39.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:39.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:39.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:39.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.087 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:39.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:39.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:39.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:39.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:39.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:39.088 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:39.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:39.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:39.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:39.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.089 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:39.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:39.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:39.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:39.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:39.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:39.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:39.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:39.090 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:39.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:39.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:39.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:39.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:39.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:39.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:39.091 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:39.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:39.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:39.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:39.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:39.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:39.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:39.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:39.092 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:39.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:39.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:39.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:39.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:39.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:39.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:39.093 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:39.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:39.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:39.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:39.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:39.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:39.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:39.094 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:39.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:39.095 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:39.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:39.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:39.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.096 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:39.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:39.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:39.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:39.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:39.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:39.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:39.097 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:39.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:39.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:39.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:39.098 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:39.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.099 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.100 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.101 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:39.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.102 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:39.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:39.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88477 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:39.103 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:39.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:39.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:39.104 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:39.107 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:39.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:39.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:39.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:39.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:39.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:39.108 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:39.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:39.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:39.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:39.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.109 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:39.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:39.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:39.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:39.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:39.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:39.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:39.110 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.111 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:39.112 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:39.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:39.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:39.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:39.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.113 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:39.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:39.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:39.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:39.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:39.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:39.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:39.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:39.114 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:39.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:39.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:39.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:39.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:39.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:39.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:39.115 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:39.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:39.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:39.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:39.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:39.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:39.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:39.116 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:39.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:39.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:39.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:39.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:39.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:39.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:39.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:39.117 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:39.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:39.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:39.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:39.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:39.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:39.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:39.118 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:39.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:39.119 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:39.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:39.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:39.120 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:39.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:39.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:39.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:39.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:39.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:39.121 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:39.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:39.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:39.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.122 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:39.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:39.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:39.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.123 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.124 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.125 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:39.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.126 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88479 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:39.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:39.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:39.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:39.128 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:39.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:39.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:39.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:39.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:39.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:39.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:39.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:39.129 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:39.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:39.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:39.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:39.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:39.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:39.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:39.131 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:39.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:39.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:39.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:39.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.132 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:39.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:39.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:39.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:39.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:39.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:39.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.133 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:39.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:39.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:39.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:39.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:39.134 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:39.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:39.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:39.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:39.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:39.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:39.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:39.136 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:39.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:39.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:39.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:39.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:39.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:39.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:39.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:39.137 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:39.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:39.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:39.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:39.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:39.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:39.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:39.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:39.138 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:39.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:39.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:39.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:39.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:39.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.139 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:39.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:39.140 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:39.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:39.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:39.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.141 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:39.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:39.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:39.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:39.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:39.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:39.142 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:39.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:39.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:39.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:39.143 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:39.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:39.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.144 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.145 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.146 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:39.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.147 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:39.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:39.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.148 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88481 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:39.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:39.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:39.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:39.149 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:39.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:39.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:39.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:39.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:39.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:39.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:39.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:39.150 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:39.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:39.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:39.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:39.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:39.151 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:39.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:39.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:39.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:39.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:39.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:39.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:39.152 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:39.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:39.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:39.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.153 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:39.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:39.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:39.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:39.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:39.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:39.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.154 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:39.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:39.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:39.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:39.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:39.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:39.155 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:39.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:39.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:39.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:39.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:39.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:39.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:39.156 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:39.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:39.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:39.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:39.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:39.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:39.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:39.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:39.157 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:39.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:39.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:39.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:39.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:39.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:39.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:39.158 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:39.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:39.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:39.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:39.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:39.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:39.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:39.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:39.159 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:39.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:39.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:39.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:39.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:39.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.160 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:39.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:39.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.161 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:39.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:39.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:39.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.162 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:39.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:39.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:39.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:39.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:39.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:39.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:39.163 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:39.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:39.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:39.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:39.164 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:39.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.165 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.166 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.167 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:39.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.168 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:39.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:39.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88484 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:39.169 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:39.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:39.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:39.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:39.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:39.170 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:39.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:39.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:39.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:39.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:39.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:39.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:39.171 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:39.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:39.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:39.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:39.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:39.172 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:39.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:39.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:39.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:39.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:39.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:39.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:39.173 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:39.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:39.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:39.174 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:39.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:39.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:39.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:39.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:39.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.175 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:39.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:39.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:39.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:39.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:39.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:39.176 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:39.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:39.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:39.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:39.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:39.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:39.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:39.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:39.177 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:39.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:39.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:39.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:39.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:39.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:39.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:39.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:39.178 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:39.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:39.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:39.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:39.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:39.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:39.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:39.179 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.181 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.183 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:39.184 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:39.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:39.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:39.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:39.185 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:39.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:39.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.186 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.187 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.188 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:39.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.189 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:39.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:39.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88485 tp_osd_tp root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:39.190 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root cwd DIR 8,1 4096 263230 /home/ubuntu/cephtest 2023-07-29T14:21:39.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root txt REG 8,1 2634560 57666 /usr/libexec/valgrind/memcheck-amd64-linux 2023-07-29T14:21:39.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 22624 57736 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-07-29T14:21:39.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 340136 57741 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-07-29T14:21:39.191 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 15752 264092 /usr/lib64/rados-classes/libcls_sdk.so.1.0.0 2023-07-29T14:21:39.192 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 326760 56727 /usr/lib64/libtcmalloc.so.4.5.9 2023-07-29T14:21:39.192 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 229320 9541 /usr/lib64/libblkid.so.1.1.0 2023-07-29T14:21:39.192 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 270800 9378 /usr/lib64/libfuse.so.2.9.9 2023-07-29T14:21:39.192 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 16448 9293 /usr/lib64/libaio.so.1.0.1 2023-07-29T14:21:39.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 49744 10227 /usr/lib64/libsnappy.so.1.1.8 2023-07-29T14:21:39.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 144120 9281 /usr/lib64/liblz4.so.1.9.3 2023-07-29T14:21:39.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 662296 10900 /usr/lib64/libcurl.so.4.7.0 2023-07-29T14:21:39.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 639224 56676 /usr/lib64/libthrift-0.15.0.so 2023-07-29T14:21:39.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:39.193 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 200136 9539 /usr/lib64/libudev.so.1.7.5 2023-07-29T14:21:39.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 137720 10013 /usr/lib64/libibverbs.so.1.14.41.0 2023-07-29T14:21:39.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 112544 2039 /usr/lib64/librdmacm.so.1.3.41.0 2023-07-29T14:21:39.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 2306496 9106 /usr/lib64/libstdc++.so.6.0.29 2023-07-29T14:21:39.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:39.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:39.194 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 61880 56717 /usr/lib64/libunwind.so.8.0.1 2023-07-29T14:21:39.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 169704 10791 /usr/lib64/libnghttp2.so.14.20.1 2023-07-29T14:21:39.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 130952 9186 /usr/lib64/libidn2.so.0.3.7 2023-07-29T14:21:39.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 479560 10896 /usr/lib64/libssh.so.4.9.4 2023-07-29T14:21:39.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 77488 9302 /usr/lib64/libpsl.so.5.3.3 2023-07-29T14:21:39.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:39.195 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:39.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:39.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 53048 9306 /usr/lib64/libbrotlidec.so.1.0.9 2023-07-29T14:21:39.196 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 600544 9234 /usr/lib64/libnl-route-3.so.200.26.0 2023-07-29T14:21:39.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 146872 9226 /usr/lib64/libnl-3.so.200.26.0 2023-07-29T14:21:39.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 1591920 9148 /usr/lib64/libunistring.so.2.1.0 2023-07-29T14:21:39.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:39.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:39.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 138712 9304 /usr/lib64/libbrotlicommon.so.1.0.9 2023-07-29T14:21:39.197 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:39.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 53272 56653 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-07-29T14:21:39.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 54392 9412 /usr/lib64/libnuma.so.1.0.0 2023-07-29T14:21:39.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 41360 11276 /usr/lib64/liburcu-bp.so.6.1.0 2023-07-29T14:21:39.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 45480 11278 /usr/lib64/liburcu-cds.so.6.1.0 2023-07-29T14:21:39.198 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 24248 11280 /usr/lib64/liburcu-common.so.6.1.0 2023-07-29T14:21:39.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 87608 264037 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-07-29T14:21:39.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 370504 264027 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-07-29T14:21:39.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 1485912 264031 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-07-29T14:21:39.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 49808 264012 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-07-29T14:21:39.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 133480 264089 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-07-29T14:21:39.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 1247032 264086 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-07-29T14:21:39.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 133768 56738 /usr/lib64/libfmt.so.8.1.1 2023-07-29T14:21:39.199 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 104032 264077 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-07-29T14:21:39.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 58056 264104 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-07-29T14:21:39.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 83248 264074 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-07-29T14:21:39.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 69736 56691 /usr/lib64/liboath.so.0.1.3 2023-07-29T14:21:39.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 426336 264080 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-07-29T14:21:39.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 58160 264047 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-07-29T14:21:39.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 66416 264065 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-07-29T14:21:39.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 58088 264098 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-07-29T14:21:39.200 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 82952 264101 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-07-29T14:21:39.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 137552 264041 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-07-29T14:21:39.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 124912 264053 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-07-29T14:21:39.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 45544 264095 /usr/lib64/rados-classes/libcls_test_remote_reads.so.1.0.0 2023-07-29T14:21:39.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 834752 264068 /usr/lib64/rados-classes/libcls_lua.so.1.0.0 2023-07-29T14:21:39.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 287248 9184 /usr/lib64/liblua-5.4.so 2023-07-29T14:21:39.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 91928 264044 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-07-29T14:21:39.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 87440 264050 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-07-29T14:21:39.201 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 54088 264056 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-07-29T14:21:39.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 49568 264071 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-07-29T14:21:39.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 100048 264059 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-07-29T14:21:39.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 877024 264083 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-07-29T14:21:39.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 91520 264062 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-07-29T14:21:39.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root mem REG 8,1 48 58787 /tmp/vgdb-pipe-shared-mem-vgdb-85182-by-root-on-??? 2023-07-29T14:21:39.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root DEL REG 0,17 183632 /[aio] 2023-07-29T14:21:39.202 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root DEL REG 0,17 186529 /[aio] 2023-07-29T14:21:39.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 1w FIFO 0,13 0t0 185404 pipe 2023-07-29T14:21:39.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 2w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 3w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 4r FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 5w FIFO 0,13 0t0 181939 pipe 2023-07-29T14:21:39.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 6u a_inode 0,14 0 2079 [eventpoll:7,20,21,22,23,24,25,26,27,59] 2023-07-29T14:21:39.203 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 7r FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 8w FIFO 0,13 0t0 181948 pipe 2023-07-29T14:21:39.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 9u a_inode 0,14 0 2079 [eventpoll:10] 2023-07-29T14:21:39.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 10r FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 11w FIFO 0,13 0t0 181949 pipe 2023-07-29T14:21:39.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 12u a_inode 0,14 0 2079 [eventpoll:13] 2023-07-29T14:21:39.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 13r FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.204 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 14w FIFO 0,13 0t0 181950 pipe 2023-07-29T14:21:39.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 15w REG 8,1 428396856 263435 /var/log/ceph/ceph-osd.7.log 2023-07-29T14:21:39.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 16uW REG 0,24 6 2094 /run/ceph/ceph-osd.7.pid 2023-07-29T14:21:39.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 17r FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 18w FIFO 0,13 0t0 180776 pipe 2023-07-29T14:21:39.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 19u unix 0xffff89b9c7b40000 0t0 180777 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-07-29T14:21:39.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 20u IPv4 187409 0t0 TCP *:6802 (LISTEN) 2023-07-29T14:21:39.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 21u IPv4 187413 0t0 TCP *:6803 (LISTEN) 2023-07-29T14:21:39.205 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 22u IPv4 203874 0t0 TCP smithi045.front.sepia.ceph.com:6804 (LISTEN) 2023-07-29T14:21:39.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 23u IPv4 203880 0t0 TCP smithi045.front.sepia.ceph.com:6805 (LISTEN) 2023-07-29T14:21:39.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 24u IPv4 187431 0t0 TCP *:6806 (LISTEN) 2023-07-29T14:21:39.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 25u IPv4 187439 0t0 TCP *:6807 (LISTEN) 2023-07-29T14:21:39.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 26u IPv4 187448 0t0 TCP *:6808 (LISTEN) 2023-07-29T14:21:39.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 27u IPv4 181967 0t0 TCP *:6809 (LISTEN) 2023-07-29T14:21:39.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 28r FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 29w FIFO 0,13 0t0 184655 pipe 2023-07-29T14:21:39.206 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 30r FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 31w FIFO 0,13 0t0 184656 pipe 2023-07-29T14:21:39.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 32u sock 0,8 0t0 184675 protocol: UDPv6 2023-07-29T14:21:39.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 33r DIR 253,3 329 64 /var/lib/ceph/osd/ceph-7 2023-07-29T14:21:39.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 34uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-07-29T14:21:39.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.207 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.208 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.209 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 58u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-07-29T14:21:39.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 59u IPv4 205887 0t0 TCP smithi045.front.sepia.ceph.com:33696->smithi045.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-07-29T14:21:39.210 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 60r FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 61w FIFO 0,13 0t0 186535 pipe 2023-07-29T14:21:39.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 62r FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root 63w FIFO 0,13 0t0 186536 pipe 2023-07-29T14:21:39.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root *524r REG 8,1 25232448 58256 /usr/bin/ceph-osd 2023-07-29T14:21:39.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root *525u REG 8,1 32 58789 /tmp/valgrind_proc_85182_cmdline_21bb4cfe (deleted) 2023-07-29T14:21:39.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root *526u REG 8,1 336 58790 /tmp/valgrind_proc_85182_auxv_21bb4cfe (deleted) 2023-07-29T14:21:39.211 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root *527w FIFO 0,13 0t0 185405 pipe 2023-07-29T14:21:39.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root *528w REG 8,1 1234 396035 /var/log/ceph/valgrind/osd.7.log 2023-07-29T14:21:39.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root *529r FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root *530w FIFO 0,13 0t0 181938 pipe 2023-07-29T14:21:39.212 INFO:teuthology.orchestra.run.smithi045.stdout:memcheck- 85182 88487 osd_srv_h root *531r FIFO 8,1 0t0 58797 /tmp/vgdb-pipe-from-vgdb-to-85182-by-root-on-??? 2023-07-29T14:21:39.212 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 88639 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.212 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 88639 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.212 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/4 88639 root txt unknown /proc/88639/exe 2023-07-29T14:21:39.213 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 88640 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.213 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 88640 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.213 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 88640 root txt unknown /proc/88640/exe 2023-07-29T14:21:39.213 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 88641 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.213 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 88641 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.213 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/7 88641 root txt unknown /proc/88641/exe 2023-07-29T14:21:39.213 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 88642 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.214 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 88642 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.214 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 88642 root txt unknown /proc/88642/exe 2023-07-29T14:21:39.214 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 88643 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.214 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 88643 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.214 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 88643 root txt unknown /proc/88643/exe 2023-07-29T14:21:39.214 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 88648 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.214 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 88648 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.215 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 88648 root txt unknown /proc/88648/exe 2023-07-29T14:21:39.215 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 89022 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.215 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 89022 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.215 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/0 89022 root txt unknown /proc/89022/exe 2023-07-29T14:21:39.215 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root cwd DIR 8,1 4096 1203 /var/spool/anacron 2023-07-29T14:21:39.215 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.215 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root txt REG 8,1 41136 10643 /usr/sbin/anacron 2023-07-29T14:21:39.215 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:39.216 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.216 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:39.216 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:39.216 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:39.216 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:39.216 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:39.217 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:39.217 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:39.217 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:39.217 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:39.217 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:39.217 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:39.217 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.217 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:39.218 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root 0r CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.218 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root 1w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.218 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root 2w CHR 1,3 0t0 4 /dev/null 2023-07-29T14:21:39.218 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root 4u unix 0xffff89b9c8cd9980 0t0 204831 type=DGRAM (CONNECTED) 2023-07-29T14:21:39.218 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root 5uW REG 8,1 9 1206 /var/spool/anacron/cron.weekly 2023-07-29T14:21:39.218 INFO:teuthology.orchestra.run.smithi045.stdout:anacron 89125 root 6uW REG 8,1 0 1207 /var/spool/anacron/cron.monthly 2023-07-29T14:21:39.218 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 89128 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.218 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 89128 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.219 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 89128 root txt unknown /proc/89128/exe 2023-07-29T14:21:39.219 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89129 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.219 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89129 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.219 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89129 root txt unknown /proc/89129/exe 2023-07-29T14:21:39.219 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 89130 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.219 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 89130 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.219 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/6 89130 root txt unknown /proc/89130/exe 2023-07-29T14:21:39.219 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 89132 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.220 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 89132 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.220 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/3 89132 root txt unknown /proc/89132/exe 2023-07-29T14:21:39.220 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89134 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.220 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89134 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.220 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89134 root txt unknown /proc/89134/exe 2023-07-29T14:21:39.220 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 89135 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.220 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 89135 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.221 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/5 89135 root txt unknown /proc/89135/exe 2023-07-29T14:21:39.221 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 89136 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.221 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 89136 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.221 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 89136 root txt unknown /proc/89136/exe 2023-07-29T14:21:39.221 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89138 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.221 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89138 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.221 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89138 root txt unknown /proc/89138/exe 2023-07-29T14:21:39.221 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89139 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.222 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89139 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.222 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/2 89139 root txt unknown /proc/89139/exe 2023-07-29T14:21:39.222 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 89159 root cwd DIR 8,1 4096 2 / 2023-07-29T14:21:39.222 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 89159 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.222 INFO:teuthology.orchestra.run.smithi045.stdout:kworker/u 89159 root txt unknown /proc/89159/exe 2023-07-29T14:21:39.222 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu cwd DIR 8,1 4096 1141 /home/ubuntu 2023-07-29T14:21:39.222 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.222 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu txt REG 8,1 1388920 9081 /usr/bin/bash 2023-07-29T14:21:39.223 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:39.223 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.223 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:39.223 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:39.223 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:39.223 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:39.223 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:39.223 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:39.224 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:39.224 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:39.224 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:39.224 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:39.224 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 191600 9079 /usr/lib64/libtinfo.so.6.2 2023-07-29T14:21:39.224 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:39.225 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.225 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:39.225 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu 0r FIFO 0,13 0t0 199373 pipe 2023-07-29T14:21:39.225 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu 1w FIFO 0,13 0t0 199374 pipe 2023-07-29T14:21:39.225 INFO:teuthology.orchestra.run.smithi045.stdout:bash 89219 ubuntu 2w FIFO 0,13 0t0 199375 pipe 2023-07-29T14:21:39.225 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root cwd DIR 8,1 4096 1141 /home/ubuntu 2023-07-29T14:21:39.225 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.226 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root txt REG 8,1 185296 11174 /usr/bin/sudo 2023-07-29T14:21:39.226 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 357408 9535 /usr/lib64/libnss_systemd.so.2 2023-07-29T14:21:39.226 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 45408 9200 /usr/lib64/libffi.so.8.1.0 2023-07-29T14:21:39.226 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 108144 3514 /usr/lib64/libgcc_s-11-20221121.so.1 2023-07-29T14:21:39.226 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 1293840 9203 /usr/lib64/libp11-kit.so.0.3.0 2023-07-29T14:21:39.226 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 36944 9117 /usr/lib64/libcap.so.2.48 2023-07-29T14:21:39.226 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 19600 10382 /usr/lib64/libpam_misc.so.0.82.1 2023-07-29T14:21:39.226 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 19400 26916 /usr/lib64/security/pam_succeed_if.so 2023-07-29T14:21:39.227 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 514272 27100 /usr/lib64/security/pam_systemd.so 2023-07-29T14:21:39.227 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 27536 26895 /usr/lib64/security/pam_limits.so 2023-07-29T14:21:39.227 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 48392 9642 /usr/lib64/libcrack.so.2.9.0 2023-07-29T14:21:39.227 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 32608 10375 /usr/lib64/libpwquality.so.1.0.2 2023-07-29T14:21:39.227 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 15352 26893 /usr/lib64/security/pam_keyinit.so 2023-07-29T14:21:39.227 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 56712 26921 /usr/lib64/security/pam_unix.so 2023-07-29T14:21:39.227 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.228 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 15840 26832 /usr/lib64/security/pam_pwquality.so 2023-07-29T14:21:39.228 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 14928 26881 /usr/lib64/security/pam_deny.so 2023-07-29T14:21:39.228 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 15184 26885 /usr/lib64/security/pam_faildelay.so 2023-07-29T14:21:39.228 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.228 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.228 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.228 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.229 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.229 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 201808 9110 /usr/lib64/libcrypt.so.2.0.0 2023-07-29T14:21:39.229 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 688232 9424 /usr/lib64/libssl.so.3.0.7 2023-07-29T14:21:39.229 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 128416 10829 /usr/lib64/libsasl2.so.3.0.0 2023-07-29T14:21:39.229 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 366424 9825 /usr/lib64/libevent-2.1.so.7.0.1 2023-07-29T14:21:39.229 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 904656 9042 /usr/lib64/libm.so.6 2023-07-29T14:21:39.229 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.229 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 2586930 14335 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-07-29T14:21:39.230 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.230 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.230 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 417696 10838 /usr/lib64/libldap_r.so.2.0.200 2023-07-29T14:21:39.230 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 472584 48691 /usr/libexec/sudo/sudoers.so 2023-07-29T14:21:39.230 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.230 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.231 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 41056 10371 /usr/lib64/libeconf.so.0.4.1 2023-07-29T14:21:39.231 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 70400 10834 /usr/lib64/liblber.so.2.0.200 2023-07-29T14:21:39.231 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 69472 10380 /usr/lib64/libpam.so.0.85.1 2023-07-29T14:21:39.231 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 19464 26883 /usr/lib64/security/pam_env.so 2023-07-29T14:21:39.232 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:39.232 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 54 14338 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-07-29T14:21:39.232 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 3284 25018 /usr/lib/locale/en_US.utf8/LC_TIME 2023-07-29T14:21:39.233 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 286 25016 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-07-29T14:21:39.233 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 57 22751 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-07-29T14:21:39.233 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 34 25068 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-07-29T14:21:39.233 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 77 14337 /usr/lib/locale/en_US.utf8/LC_NAME 2023-07-29T14:21:39.234 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 167 25011 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-07-29T14:21:39.234 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 59 25017 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-07-29T14:21:39.234 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 23 25014 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-07-29T14:21:39.234 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 32528 9137 /usr/lib64/libcap-ng.so.0.0.0 2023-07-29T14:21:39.235 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 119832 48687 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-07-29T14:21:39.235 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.235 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 135272 9142 /usr/lib64/libaudit.so.1.0.0 2023-07-29T14:21:39.235 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:39.236 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.236 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root mem REG 8,1 369 25013 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-07-29T14:21:39.236 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root 0r FIFO 0,13 0t0 199373 pipe 2023-07-29T14:21:39.237 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root 1w FIFO 0,13 0t0 199374 pipe 2023-07-29T14:21:39.237 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root 2w FIFO 0,13 0t0 199375 pipe 2023-07-29T14:21:39.237 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root 3r FIFO 0,13 0t0 202230 pipe 2023-07-29T14:21:39.238 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root 4w FIFO 0,13 0t0 202230 pipe 2023-07-29T14:21:39.238 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root 5u netlink 0t0 202236 AUDIT 2023-07-29T14:21:39.238 INFO:teuthology.orchestra.run.smithi045.stdout:sudo 89235 root 6u unix 0xffff89b9c8312ec0 0t0 202237 type=DGRAM (CONNECTED) 2023-07-29T14:21:39.238 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root cwd DIR 8,1 4096 1141 /home/ubuntu 2023-07-29T14:21:39.239 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.239 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root txt REG 8,1 179232 11757 /usr/bin/lsof 2023-07-29T14:21:39.239 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.239 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:39.240 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.240 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.240 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.240 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.241 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.241 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.241 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.241 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.242 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.242 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.242 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 196120 10846 /usr/lib64/libtirpc.so.3.0.0 2023-07-29T14:21:39.243 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.243 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:39.243 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.243 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root 0r FIFO 0,13 0t0 199373 pipe 2023-07-29T14:21:39.243 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root 1w FIFO 0,13 0t0 199374 pipe 2023-07-29T14:21:39.244 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root 2w FIFO 0,13 0t0 199375 pipe 2023-07-29T14:21:39.244 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root 3r DIR 0,20 0 1 /proc 2023-07-29T14:21:39.244 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root 4r DIR 0,20 0 203113 /proc/89237/fd 2023-07-29T14:21:39.244 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root 5w FIFO 0,13 0t0 203118 pipe 2023-07-29T14:21:39.244 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89237 root 6r FIFO 0,13 0t0 203119 pipe 2023-07-29T14:21:39.244 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root cwd DIR 8,1 4096 1141 /home/ubuntu 2023-07-29T14:21:39.245 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root rtd DIR 8,1 4096 2 / 2023-07-29T14:21:39.245 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root txt REG 8,1 179232 11757 /usr/bin/lsof 2023-07-29T14:21:39.245 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 4474688 9422 /usr/lib64/libcrypto.so.3.0.7 2023-07-29T14:21:39.245 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 346132 25099 /usr/lib/locale/C.utf8/LC_CTYPE 2023-07-29T14:21:39.245 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 920296 10818 /usr/lib64/libkrb5.so.3.3 2023-07-29T14:21:39.245 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 2387200 9039 /usr/lib64/libc.so.6 2023-07-29T14:21:39.245 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 102568 9101 /usr/lib64/libz.so.1.2.11 2023-07-29T14:21:39.245 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 70512 9048 /usr/lib64/libresolv.so.2 2023-07-29T14:21:39.246 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 23992 9174 /usr/lib64/libkeyutils.so.1.10 2023-07-29T14:21:39.246 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 66896 10820 /usr/lib64/libkrb5support.so.0.1 2023-07-29T14:21:39.246 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 24400 9125 /usr/lib64/libcom_err.so.2.1 2023-07-29T14:21:39.246 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 100488 10812 /usr/lib64/libk5crypto.so.3.1 2023-07-29T14:21:39.246 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 359680 10808 /usr/lib64/libgssapi_krb5.so.2.2 2023-07-29T14:21:39.246 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 637872 9154 /usr/lib64/libpcre2-8.so.0.11.0 2023-07-29T14:21:39.246 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 196120 10846 /usr/lib64/libtirpc.so.3.0.0 2023-07-29T14:21:39.247 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 175592 9239 /usr/lib64/libselinux.so.1 2023-07-29T14:21:39.247 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 26988 25176 /usr/lib64/gconv/gconv-modules.cache 2023-07-29T14:21:39.247 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root mem REG 8,1 842848 9035 /usr/lib64/ld-linux-x86-64.so.2 2023-07-29T14:21:39.247 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root 4r FIFO 0,13 0t0 203118 pipe 2023-07-29T14:21:39.247 INFO:teuthology.orchestra.run.smithi045.stdout:lsof 89238 root 7w FIFO 0,13 0t0 203119 pipe 2023-07-29T14:21:39.247 INFO:teuthology.orchestra.run.smithi045.stdout:USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 2023-07-29T14:21:39.247 INFO:teuthology.orchestra.run.smithi045.stdout:root 2 0.0 0.0 0 0 ? S 13:09 0:00 [kthreadd] 2023-07-29T14:21:39.248 INFO:teuthology.orchestra.run.smithi045.stdout:root 3 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [rcu_gp] 2023-07-29T14:21:39.248 INFO:teuthology.orchestra.run.smithi045.stdout:root 4 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [rcu_par_gp] 2023-07-29T14:21:39.248 INFO:teuthology.orchestra.run.smithi045.stdout:root 5 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [slub_flushwq] 2023-07-29T14:21:39.248 INFO:teuthology.orchestra.run.smithi045.stdout:root 6 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [netns] 2023-07-29T14:21:39.248 INFO:teuthology.orchestra.run.smithi045.stdout:root 8 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/0:0H-events_highpri] 2023-07-29T14:21:39.248 INFO:teuthology.orchestra.run.smithi045.stdout:root 10 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [mm_percpu_wq] 2023-07-29T14:21:39.248 INFO:teuthology.orchestra.run.smithi045.stdout:root 12 0.0 0.0 0 0 ? I 13:09 0:00 \_ [rcu_tasks_kthre] 2023-07-29T14:21:39.248 INFO:teuthology.orchestra.run.smithi045.stdout:root 13 0.0 0.0 0 0 ? I 13:09 0:00 \_ [rcu_tasks_rude_] 2023-07-29T14:21:39.249 INFO:teuthology.orchestra.run.smithi045.stdout:root 14 0.0 0.0 0 0 ? I 13:09 0:00 \_ [rcu_tasks_trace] 2023-07-29T14:21:39.249 INFO:teuthology.orchestra.run.smithi045.stdout:root 15 0.0 0.0 0 0 ? S 13:09 0:00 \_ [ksoftirqd/0] 2023-07-29T14:21:39.249 INFO:teuthology.orchestra.run.smithi045.stdout:root 16 0.1 0.0 0 0 ? S 13:09 0:04 \_ [pr/ttyS1] 2023-07-29T14:21:39.249 INFO:teuthology.orchestra.run.smithi045.stdout:root 17 0.0 0.0 0 0 ? I 13:09 0:00 \_ [rcu_preempt] 2023-07-29T14:21:39.249 INFO:teuthology.orchestra.run.smithi045.stdout:root 18 0.0 0.0 0 0 ? S 13:09 0:00 \_ [migration/0] 2023-07-29T14:21:39.249 INFO:teuthology.orchestra.run.smithi045.stdout:root 19 0.0 0.0 0 0 ? S 13:09 0:00 \_ [idle_inject/0] 2023-07-29T14:21:39.249 INFO:teuthology.orchestra.run.smithi045.stdout:root 21 0.0 0.0 0 0 ? S 13:09 0:00 \_ [cpuhp/0] 2023-07-29T14:21:39.249 INFO:teuthology.orchestra.run.smithi045.stdout:root 22 0.0 0.0 0 0 ? S 13:09 0:00 \_ [cpuhp/1] 2023-07-29T14:21:39.250 INFO:teuthology.orchestra.run.smithi045.stdout:root 23 0.0 0.0 0 0 ? S 13:09 0:00 \_ [idle_inject/1] 2023-07-29T14:21:39.250 INFO:teuthology.orchestra.run.smithi045.stdout:root 24 0.0 0.0 0 0 ? S 13:09 0:00 \_ [migration/1] 2023-07-29T14:21:39.250 INFO:teuthology.orchestra.run.smithi045.stdout:root 25 0.0 0.0 0 0 ? S 13:09 0:00 \_ [ksoftirqd/1] 2023-07-29T14:21:39.250 INFO:teuthology.orchestra.run.smithi045.stdout:root 27 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/1:0H-events_highpri] 2023-07-29T14:21:39.250 INFO:teuthology.orchestra.run.smithi045.stdout:root 28 0.0 0.0 0 0 ? S 13:09 0:00 \_ [cpuhp/2] 2023-07-29T14:21:39.250 INFO:teuthology.orchestra.run.smithi045.stdout:root 29 0.0 0.0 0 0 ? S 13:09 0:00 \_ [idle_inject/2] 2023-07-29T14:21:39.250 INFO:teuthology.orchestra.run.smithi045.stdout:root 30 0.0 0.0 0 0 ? S 13:09 0:00 \_ [migration/2] 2023-07-29T14:21:39.251 INFO:teuthology.orchestra.run.smithi045.stdout:root 31 0.0 0.0 0 0 ? S 13:09 0:00 \_ [ksoftirqd/2] 2023-07-29T14:21:39.251 INFO:teuthology.orchestra.run.smithi045.stdout:root 33 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/2:0H-events_highpri] 2023-07-29T14:21:39.251 INFO:teuthology.orchestra.run.smithi045.stdout:root 34 0.0 0.0 0 0 ? S 13:09 0:00 \_ [cpuhp/3] 2023-07-29T14:21:39.251 INFO:teuthology.orchestra.run.smithi045.stdout:root 35 0.0 0.0 0 0 ? S 13:09 0:00 \_ [idle_inject/3] 2023-07-29T14:21:39.251 INFO:teuthology.orchestra.run.smithi045.stdout:root 36 0.0 0.0 0 0 ? S 13:09 0:00 \_ [migration/3] 2023-07-29T14:21:39.251 INFO:teuthology.orchestra.run.smithi045.stdout:root 37 0.0 0.0 0 0 ? S 13:09 0:00 \_ [ksoftirqd/3] 2023-07-29T14:21:39.251 INFO:teuthology.orchestra.run.smithi045.stdout:root 39 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/3:0H-events_highpri] 2023-07-29T14:21:39.251 INFO:teuthology.orchestra.run.smithi045.stdout:root 40 0.0 0.0 0 0 ? S 13:09 0:00 \_ [cpuhp/4] 2023-07-29T14:21:39.252 INFO:teuthology.orchestra.run.smithi045.stdout:root 41 0.0 0.0 0 0 ? S 13:09 0:00 \_ [idle_inject/4] 2023-07-29T14:21:39.252 INFO:teuthology.orchestra.run.smithi045.stdout:root 42 0.0 0.0 0 0 ? S 13:09 0:00 \_ [migration/4] 2023-07-29T14:21:39.252 INFO:teuthology.orchestra.run.smithi045.stdout:root 43 0.0 0.0 0 0 ? S 13:09 0:00 \_ [ksoftirqd/4] 2023-07-29T14:21:39.252 INFO:teuthology.orchestra.run.smithi045.stdout:root 44 0.0 0.0 0 0 ? I 13:09 0:02 \_ [kworker/4:0-dio/dm-3] 2023-07-29T14:21:39.252 INFO:teuthology.orchestra.run.smithi045.stdout:root 45 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/4:0H-events_highpri] 2023-07-29T14:21:39.252 INFO:teuthology.orchestra.run.smithi045.stdout:root 46 0.0 0.0 0 0 ? S 13:09 0:00 \_ [cpuhp/5] 2023-07-29T14:21:39.252 INFO:teuthology.orchestra.run.smithi045.stdout:root 47 0.0 0.0 0 0 ? S 13:09 0:00 \_ [idle_inject/5] 2023-07-29T14:21:39.252 INFO:teuthology.orchestra.run.smithi045.stdout:root 48 0.0 0.0 0 0 ? S 13:09 0:00 \_ [migration/5] 2023-07-29T14:21:39.253 INFO:teuthology.orchestra.run.smithi045.stdout:root 49 0.0 0.0 0 0 ? S 13:09 0:00 \_ [ksoftirqd/5] 2023-07-29T14:21:39.253 INFO:teuthology.orchestra.run.smithi045.stdout:root 51 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/5:0H-events_highpri] 2023-07-29T14:21:39.253 INFO:teuthology.orchestra.run.smithi045.stdout:root 52 0.0 0.0 0 0 ? S 13:09 0:00 \_ [cpuhp/6] 2023-07-29T14:21:39.253 INFO:teuthology.orchestra.run.smithi045.stdout:root 53 0.0 0.0 0 0 ? S 13:09 0:00 \_ [idle_inject/6] 2023-07-29T14:21:39.253 INFO:teuthology.orchestra.run.smithi045.stdout:root 54 0.0 0.0 0 0 ? S 13:09 0:00 \_ [migration/6] 2023-07-29T14:21:39.253 INFO:teuthology.orchestra.run.smithi045.stdout:root 55 0.0 0.0 0 0 ? S 13:09 0:00 \_ [ksoftirqd/6] 2023-07-29T14:21:39.254 INFO:teuthology.orchestra.run.smithi045.stdout:root 57 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/6:0H-events_highpri] 2023-07-29T14:21:39.254 INFO:teuthology.orchestra.run.smithi045.stdout:root 58 0.0 0.0 0 0 ? S 13:09 0:00 \_ [cpuhp/7] 2023-07-29T14:21:39.254 INFO:teuthology.orchestra.run.smithi045.stdout:root 59 0.0 0.0 0 0 ? S 13:09 0:00 \_ [idle_inject/7] 2023-07-29T14:21:39.254 INFO:teuthology.orchestra.run.smithi045.stdout:root 60 0.0 0.0 0 0 ? S 13:09 0:00 \_ [migration/7] 2023-07-29T14:21:39.254 INFO:teuthology.orchestra.run.smithi045.stdout:root 61 0.0 0.0 0 0 ? S 13:09 0:00 \_ [ksoftirqd/7] 2023-07-29T14:21:39.254 INFO:teuthology.orchestra.run.smithi045.stdout:root 63 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/7:0H-events_highpri] 2023-07-29T14:21:39.254 INFO:teuthology.orchestra.run.smithi045.stdout:root 67 0.0 0.0 0 0 ? I 13:09 0:00 \_ [kworker/u16:4-events_unbound] 2023-07-29T14:21:39.254 INFO:teuthology.orchestra.run.smithi045.stdout:root 70 0.0 0.0 0 0 ? I 13:09 0:01 \_ [kworker/u16:7-writeback] 2023-07-29T14:21:39.255 INFO:teuthology.orchestra.run.smithi045.stdout:root 71 0.0 0.0 0 0 ? S 13:09 0:00 \_ [kdevtmpfs] 2023-07-29T14:21:39.255 INFO:teuthology.orchestra.run.smithi045.stdout:root 72 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [inet_frag_wq] 2023-07-29T14:21:39.255 INFO:teuthology.orchestra.run.smithi045.stdout:root 73 0.0 0.0 0 0 ? S 13:09 0:00 \_ [kauditd] 2023-07-29T14:21:39.255 INFO:teuthology.orchestra.run.smithi045.stdout:root 75 0.0 0.0 0 0 ? S 13:09 0:00 \_ [khungtaskd] 2023-07-29T14:21:39.255 INFO:teuthology.orchestra.run.smithi045.stdout:root 76 0.0 0.0 0 0 ? S 13:09 0:00 \_ [oom_reaper] 2023-07-29T14:21:39.255 INFO:teuthology.orchestra.run.smithi045.stdout:root 77 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [writeback] 2023-07-29T14:21:39.255 INFO:teuthology.orchestra.run.smithi045.stdout:root 78 0.0 0.0 0 0 ? S 13:09 0:00 \_ [kcompactd0] 2023-07-29T14:21:39.255 INFO:teuthology.orchestra.run.smithi045.stdout:root 79 0.0 0.0 0 0 ? SN 13:09 0:00 \_ [ksmd] 2023-07-29T14:21:39.256 INFO:teuthology.orchestra.run.smithi045.stdout:root 80 0.0 0.0 0 0 ? SN 13:09 0:00 \_ [khugepaged] 2023-07-29T14:21:39.256 INFO:teuthology.orchestra.run.smithi045.stdout:root 81 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [cryptd] 2023-07-29T14:21:39.256 INFO:teuthology.orchestra.run.smithi045.stdout:root 82 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kintegrityd] 2023-07-29T14:21:39.256 INFO:teuthology.orchestra.run.smithi045.stdout:root 83 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kblockd] 2023-07-29T14:21:39.256 INFO:teuthology.orchestra.run.smithi045.stdout:root 84 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [blkcg_punt_bio] 2023-07-29T14:21:39.256 INFO:teuthology.orchestra.run.smithi045.stdout:root 86 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [tpm_dev_wq] 2023-07-29T14:21:39.256 INFO:teuthology.orchestra.run.smithi045.stdout:root 87 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [md] 2023-07-29T14:21:39.257 INFO:teuthology.orchestra.run.smithi045.stdout:root 88 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [edac-poller] 2023-07-29T14:21:39.257 INFO:teuthology.orchestra.run.smithi045.stdout:root 89 0.0 0.0 0 0 ? S 13:09 0:00 \_ [watchdogd] 2023-07-29T14:21:39.257 INFO:teuthology.orchestra.run.smithi045.stdout:root 90 0.0 0.0 0 0 ? I 13:09 0:00 \_ [kworker/1:1-cgroup_destroy] 2023-07-29T14:21:39.257 INFO:teuthology.orchestra.run.smithi045.stdout:root 91 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/2:1H-kblockd] 2023-07-29T14:21:39.257 INFO:teuthology.orchestra.run.smithi045.stdout:root 93 0.0 0.0 0 0 ? S 13:09 0:00 \_ [kswapd0] 2023-07-29T14:21:39.257 INFO:teuthology.orchestra.run.smithi045.stdout:root 99 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kthrotld] 2023-07-29T14:21:39.257 INFO:teuthology.orchestra.run.smithi045.stdout:root 108 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [acpi_thermal_pm] 2023-07-29T14:21:39.258 INFO:teuthology.orchestra.run.smithi045.stdout:root 109 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kmpath_rdacd] 2023-07-29T14:21:39.258 INFO:teuthology.orchestra.run.smithi045.stdout:root 110 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kaluad] 2023-07-29T14:21:39.258 INFO:teuthology.orchestra.run.smithi045.stdout:root 111 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [mld] 2023-07-29T14:21:39.258 INFO:teuthology.orchestra.run.smithi045.stdout:root 112 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/5:1H-kblockd] 2023-07-29T14:21:39.258 INFO:teuthology.orchestra.run.smithi045.stdout:root 113 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [ipv6_addrconf] 2023-07-29T14:21:39.258 INFO:teuthology.orchestra.run.smithi045.stdout:root 122 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kstrp] 2023-07-29T14:21:39.258 INFO:teuthology.orchestra.run.smithi045.stdout:root 136 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [zswap-shrink] 2023-07-29T14:21:39.258 INFO:teuthology.orchestra.run.smithi045.stdout:root 250 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/u17:0] 2023-07-29T14:21:39.259 INFO:teuthology.orchestra.run.smithi045.stdout:root 300 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/4:1H-kblockd] 2023-07-29T14:21:39.259 INFO:teuthology.orchestra.run.smithi045.stdout:root 303 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/1:1H-kblockd] 2023-07-29T14:21:39.259 INFO:teuthology.orchestra.run.smithi045.stdout:root 349 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/3:1H-kblockd] 2023-07-29T14:21:39.259 INFO:teuthology.orchestra.run.smithi045.stdout:root 365 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/7:1H-kblockd] 2023-07-29T14:21:39.259 INFO:teuthology.orchestra.run.smithi045.stdout:root 388 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/6:1H-kblockd] 2023-07-29T14:21:39.259 INFO:teuthology.orchestra.run.smithi045.stdout:root 422 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [kworker/0:1H-kblockd] 2023-07-29T14:21:39.259 INFO:teuthology.orchestra.run.smithi045.stdout:root 440 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [ata_sff] 2023-07-29T14:21:39.259 INFO:teuthology.orchestra.run.smithi045.stdout:root 441 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [nvme-wq] 2023-07-29T14:21:39.260 INFO:teuthology.orchestra.run.smithi045.stdout:root 443 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [nvme-reset-wq] 2023-07-29T14:21:39.260 INFO:teuthology.orchestra.run.smithi045.stdout:root 444 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [nvme-delete-wq] 2023-07-29T14:21:39.260 INFO:teuthology.orchestra.run.smithi045.stdout:root 445 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [nvme-auth-wq] 2023-07-29T14:21:39.260 INFO:teuthology.orchestra.run.smithi045.stdout:root 451 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [ixgbe] 2023-07-29T14:21:39.260 INFO:teuthology.orchestra.run.smithi045.stdout:root 454 0.0 0.0 0 0 ? S 13:09 0:00 \_ [scsi_eh_0] 2023-07-29T14:21:39.260 INFO:teuthology.orchestra.run.smithi045.stdout:root 455 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [scsi_tmf_0] 2023-07-29T14:21:39.260 INFO:teuthology.orchestra.run.smithi045.stdout:root 456 0.0 0.0 0 0 ? S 13:09 0:00 \_ [scsi_eh_1] 2023-07-29T14:21:39.261 INFO:teuthology.orchestra.run.smithi045.stdout:root 457 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [scsi_tmf_1] 2023-07-29T14:21:39.261 INFO:teuthology.orchestra.run.smithi045.stdout:root 458 0.0 0.0 0 0 ? S 13:09 0:00 \_ [scsi_eh_2] 2023-07-29T14:21:39.261 INFO:teuthology.orchestra.run.smithi045.stdout:root 459 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [scsi_tmf_2] 2023-07-29T14:21:39.261 INFO:teuthology.orchestra.run.smithi045.stdout:root 460 0.0 0.0 0 0 ? S 13:09 0:00 \_ [scsi_eh_3] 2023-07-29T14:21:39.261 INFO:teuthology.orchestra.run.smithi045.stdout:root 461 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [scsi_tmf_3] 2023-07-29T14:21:39.261 INFO:teuthology.orchestra.run.smithi045.stdout:root 466 0.0 0.0 0 0 ? S 13:09 0:00 \_ [scsi_eh_4] 2023-07-29T14:21:39.261 INFO:teuthology.orchestra.run.smithi045.stdout:root 467 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [scsi_tmf_4] 2023-07-29T14:21:39.262 INFO:teuthology.orchestra.run.smithi045.stdout:root 468 0.0 0.0 0 0 ? S 13:09 0:00 \_ [scsi_eh_5] 2023-07-29T14:21:39.262 INFO:teuthology.orchestra.run.smithi045.stdout:root 469 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [scsi_tmf_5] 2023-07-29T14:21:39.262 INFO:teuthology.orchestra.run.smithi045.stdout:root 470 0.0 0.0 0 0 ? S 13:09 0:00 \_ [scsi_eh_6] 2023-07-29T14:21:39.262 INFO:teuthology.orchestra.run.smithi045.stdout:root 471 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [scsi_tmf_6] 2023-07-29T14:21:39.262 INFO:teuthology.orchestra.run.smithi045.stdout:root 472 0.0 0.0 0 0 ? S 13:09 0:00 \_ [scsi_eh_7] 2023-07-29T14:21:39.263 INFO:teuthology.orchestra.run.smithi045.stdout:root 473 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [scsi_tmf_7] 2023-07-29T14:21:39.263 INFO:teuthology.orchestra.run.smithi045.stdout:root 474 0.0 0.0 0 0 ? S 13:09 0:00 \_ [scsi_eh_8] 2023-07-29T14:21:39.263 INFO:teuthology.orchestra.run.smithi045.stdout:root 475 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [scsi_tmf_8] 2023-07-29T14:21:39.263 INFO:teuthology.orchestra.run.smithi045.stdout:root 476 0.0 0.0 0 0 ? S 13:09 0:00 \_ [scsi_eh_9] 2023-07-29T14:21:39.264 INFO:teuthology.orchestra.run.smithi045.stdout:root 477 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [scsi_tmf_9] 2023-07-29T14:21:39.264 INFO:teuthology.orchestra.run.smithi045.stdout:root 505 0.0 0.0 0 0 ? S 13:09 0:00 \_ [jbd2/sda1-8] 2023-07-29T14:21:39.264 INFO:teuthology.orchestra.run.smithi045.stdout:root 506 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [ext4-rsv-conver] 2023-07-29T14:21:39.265 INFO:teuthology.orchestra.run.smithi045.stdout:root 581 0.0 0.0 0 0 ? S 13:09 0:00 \_ [ext4lazyinit] 2023-07-29T14:21:39.265 INFO:teuthology.orchestra.run.smithi045.stdout:root 645 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [ipmi-msghandler] 2023-07-29T14:21:39.265 INFO:teuthology.orchestra.run.smithi045.stdout:root 654 0.0 0.0 0 0 ? SN 13:09 0:00 \_ [kipmi0] 2023-07-29T14:21:39.265 INFO:teuthology.orchestra.run.smithi045.stdout:root 670 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [rpciod] 2023-07-29T14:21:39.265 INFO:teuthology.orchestra.run.smithi045.stdout:root 671 0.0 0.0 0 0 ? I< 13:09 0:00 \_ [xprtiod] 2023-07-29T14:21:39.266 INFO:teuthology.orchestra.run.smithi045.stdout:root 58231 0.0 0.0 0 0 ? I< 13:14 0:00 \_ [tls-strp] 2023-07-29T14:21:39.266 INFO:teuthology.orchestra.run.smithi045.stdout:root 68085 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [kdmflush/253:0] 2023-07-29T14:21:39.266 INFO:teuthology.orchestra.run.smithi045.stdout:root 68205 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [kdmflush/253:1] 2023-07-29T14:21:39.266 INFO:teuthology.orchestra.run.smithi045.stdout:root 68325 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [kdmflush/253:2] 2023-07-29T14:21:39.267 INFO:teuthology.orchestra.run.smithi045.stdout:root 68445 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [kdmflush/253:3] 2023-07-29T14:21:39.267 INFO:teuthology.orchestra.run.smithi045.stdout:root 68565 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [kdmflush/253:4] 2023-07-29T14:21:39.267 INFO:teuthology.orchestra.run.smithi045.stdout:root 69458 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [xfsalloc] 2023-07-29T14:21:39.267 INFO:teuthology.orchestra.run.smithi045.stdout:root 69459 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [xfs_mru_cache] 2023-07-29T14:21:39.268 INFO:teuthology.orchestra.run.smithi045.stdout:root 69460 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [xfs-buf/dm-4] 2023-07-29T14:21:39.268 INFO:teuthology.orchestra.run.smithi045.stdout:root 69461 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [xfs-conv/dm-4] 2023-07-29T14:21:39.268 INFO:teuthology.orchestra.run.smithi045.stdout:root 69462 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [xfs-reclaim/dm-] 2023-07-29T14:21:39.268 INFO:teuthology.orchestra.run.smithi045.stdout:root 69463 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [xfs-blockgc/dm-] 2023-07-29T14:21:39.269 INFO:teuthology.orchestra.run.smithi045.stdout:root 69464 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [xfs-inodegc/dm-] 2023-07-29T14:21:39.269 INFO:teuthology.orchestra.run.smithi045.stdout:root 69465 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [xfs-log/dm-4] 2023-07-29T14:21:39.269 INFO:teuthology.orchestra.run.smithi045.stdout:root 69466 0.0 0.0 0 0 ? I< 13:16 0:00 \_ [xfs-cil/dm-4] 2023-07-29T14:21:39.269 INFO:teuthology.orchestra.run.smithi045.stdout:root 69467 0.0 0.0 0 0 ? S 13:16 0:00 \_ [xfsaild/dm-4] 2023-07-29T14:21:39.270 INFO:teuthology.orchestra.run.smithi045.stdout:root 72478 0.0 0.0 0 0 ? I 13:16 0:00 \_ [kworker/1:2-mm_percpu_wq] 2023-07-29T14:21:39.270 INFO:teuthology.orchestra.run.smithi045.stdout:root 75511 0.0 0.0 0 0 ? I 13:18 0:00 \_ [kworker/u16:2-flush-8:0] 2023-07-29T14:21:39.270 INFO:teuthology.orchestra.run.smithi045.stdout:root 77622 0.0 0.0 0 0 ? I 13:20 0:00 \_ [kworker/7:3-mm_percpu_wq] 2023-07-29T14:21:39.270 INFO:teuthology.orchestra.run.smithi045.stdout:root 79386 0.0 0.0 0 0 ? I< 13:26 0:00 \_ [xfs-buf/dm-3] 2023-07-29T14:21:39.271 INFO:teuthology.orchestra.run.smithi045.stdout:root 79387 0.0 0.0 0 0 ? I< 13:26 0:00 \_ [xfs-conv/dm-3] 2023-07-29T14:21:39.271 INFO:teuthology.orchestra.run.smithi045.stdout:root 79388 0.0 0.0 0 0 ? I< 13:26 0:00 \_ [xfs-reclaim/dm-] 2023-07-29T14:21:39.271 INFO:teuthology.orchestra.run.smithi045.stdout:root 79389 0.0 0.0 0 0 ? I< 13:26 0:00 \_ [xfs-blockgc/dm-] 2023-07-29T14:21:39.272 INFO:teuthology.orchestra.run.smithi045.stdout:root 79390 0.0 0.0 0 0 ? I< 13:26 0:00 \_ [xfs-inodegc/dm-] 2023-07-29T14:21:39.272 INFO:teuthology.orchestra.run.smithi045.stdout:root 79391 0.0 0.0 0 0 ? I< 13:26 0:00 \_ [xfs-log/dm-3] 2023-07-29T14:21:39.272 INFO:teuthology.orchestra.run.smithi045.stdout:root 79392 0.0 0.0 0 0 ? I< 13:26 0:00 \_ [xfs-cil/dm-3] 2023-07-29T14:21:39.273 INFO:teuthology.orchestra.run.smithi045.stdout:root 79393 0.0 0.0 0 0 ? S 13:26 0:00 \_ [xfsaild/dm-3] 2023-07-29T14:21:39.273 INFO:teuthology.orchestra.run.smithi045.stdout:root 82960 0.0 0.0 0 0 ? I< 13:26 0:00 \_ [dio/dm-3] 2023-07-29T14:21:39.273 INFO:teuthology.orchestra.run.smithi045.stdout:root 88639 0.1 0.0 0 0 ? I 13:40 0:02 \_ [kworker/4:1-events] 2023-07-29T14:21:39.273 INFO:teuthology.orchestra.run.smithi045.stdout:root 88640 0.0 0.0 0 0 ? I 13:40 0:00 \_ [kworker/0:2-kacpi_notify] 2023-07-29T14:21:39.274 INFO:teuthology.orchestra.run.smithi045.stdout:root 88641 0.0 0.0 0 0 ? I 13:40 0:00 \_ [kworker/7:0-dio/dm-3] 2023-07-29T14:21:39.274 INFO:teuthology.orchestra.run.smithi045.stdout:root 88642 0.0 0.0 0 0 ? I 13:40 0:00 \_ [kworker/3:1-mm_percpu_wq] 2023-07-29T14:21:39.274 INFO:teuthology.orchestra.run.smithi045.stdout:root 88643 0.0 0.0 0 0 ? I 13:40 0:00 \_ [kworker/u16:0-flush-8:0] 2023-07-29T14:21:39.274 INFO:teuthology.orchestra.run.smithi045.stdout:root 88648 0.0 0.0 0 0 ? I 13:49 0:00 \_ [kworker/5:2-mm_percpu_wq] 2023-07-29T14:21:39.275 INFO:teuthology.orchestra.run.smithi045.stdout:root 89022 0.0 0.0 0 0 ? I 13:55 0:00 \_ [kworker/0:0] 2023-07-29T14:21:39.275 INFO:teuthology.orchestra.run.smithi045.stdout:root 89128 0.0 0.0 0 0 ? I 14:01 0:00 \_ [kworker/6:1-mm_percpu_wq] 2023-07-29T14:21:39.275 INFO:teuthology.orchestra.run.smithi045.stdout:root 89129 0.0 0.0 0 0 ? I 14:01 0:00 \_ [kworker/2:1-cgwb_release] 2023-07-29T14:21:39.275 INFO:teuthology.orchestra.run.smithi045.stdout:root 89130 0.0 0.0 0 0 ? I 14:08 0:00 \_ [kworker/6:0-mm_percpu_wq] 2023-07-29T14:21:39.276 INFO:teuthology.orchestra.run.smithi045.stdout:root 89132 0.0 0.0 0 0 ? I 14:10 0:00 \_ [kworker/3:0-cgroup_destroy] 2023-07-29T14:21:39.276 INFO:teuthology.orchestra.run.smithi045.stdout:root 89134 0.0 0.0 0 0 ? I 14:12 0:00 \_ [kworker/2:2-events_freezable] 2023-07-29T14:21:39.276 INFO:teuthology.orchestra.run.smithi045.stdout:root 89135 0.0 0.0 0 0 ? I 14:13 0:00 \_ [kworker/5:1-mm_percpu_wq] 2023-07-29T14:21:39.277 INFO:teuthology.orchestra.run.smithi045.stdout:root 89136 0.0 0.0 0 0 ? I 14:18 0:00 \_ [kworker/u16:1-flush-8:0] 2023-07-29T14:21:39.277 INFO:teuthology.orchestra.run.smithi045.stdout:root 89138 0.0 0.0 0 0 ? I 14:20 0:00 \_ [kworker/2:0-cgroup_destroy] 2023-07-29T14:21:39.277 INFO:teuthology.orchestra.run.smithi045.stdout:root 89139 0.0 0.0 0 0 ? I 14:20 0:00 \_ [kworker/2:3] 2023-07-29T14:21:39.277 INFO:teuthology.orchestra.run.smithi045.stdout:root 89159 0.0 0.0 0 0 ? I 14:21 0:00 \_ [kworker/u16:3-flush-8:0] 2023-07-29T14:21:39.278 INFO:teuthology.orchestra.run.smithi045.stdout:root 1 0.0 0.0 175952 20276 ? Ss 13:09 0:03 /usr/lib/systemd/systemd --switched-root --system --deserialize 31 2023-07-29T14:21:39.278 INFO:teuthology.orchestra.run.smithi045.stdout:root 575 0.0 0.1 67564 44264 ? Ss 13:09 0:02 /usr/lib/systemd/systemd-journald 2023-07-29T14:21:39.278 INFO:teuthology.orchestra.run.smithi045.stdout:root 594 0.0 0.0 37476 15616 ? Ss 13:09 0:00 /usr/lib/systemd/systemd-udevd 2023-07-29T14:21:39.278 INFO:teuthology.orchestra.run.smithi045.stdout:rpc 669 0.0 0.0 13264 5556 ? Ss 13:09 0:00 /usr/bin/rpcbind -w -f 2023-07-29T14:21:39.279 INFO:teuthology.orchestra.run.smithi045.stdout:root 672 0.0 0.0 91828 4460 ? S sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-29T14:21:39.293 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:37.299+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:39.293 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:38.351+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:39.296 DEBUG:teuthology.orchestra.run.smithi045:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-29T14:21:39.348 INFO:teuthology.orchestra.run.smithi040.stdout:/var/log/ceph/valgrind/mon.a.log: Leak_StillReachable 2023-07-29T14:21:39.349 INFO:teuthology.orchestra.run.smithi040.stdout:/var/log/ceph/valgrind/mon.c.log: Leak_StillReachable 2023-07-29T14:21:39.349 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mon.a.log kind Leak_StillReachable 2023-07-29T14:21:39.350 ERROR:tasks.ceph:saw valgrind issue Leak_StillReachable in /var/log/ceph/valgrind/mon.a.log 2023-07-29T14:21:39.350 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mon.c.log kind Leak_StillReachable 2023-07-29T14:21:39.351 ERROR:tasks.ceph:saw valgrind issue Leak_StillReachable in /var/log/ceph/valgrind/mon.c.log 2023-07-29T14:21:39.352 INFO:tasks.ceph:Archiving crash dumps... 2023-07-29T14:21:39.355 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625/remote/smithi040/crash 2023-07-29T14:21:39.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-29T14:21:39.370 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:39.377+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:39.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~260s 2023-07-29T14:21:39.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-07-29T14:21:39.401 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625/remote/smithi045/crash 2023-07-29T14:21:39.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-29T14:21:39.455 INFO:tasks.ceph:Compressing logs... 2023-07-29T14:21:39.456 DEBUG:teuthology.orchestra.run.smithi040:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-29T14:21:39.459 DEBUG:teuthology.orchestra.run.smithi045:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-29T14:21:40.408 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:40.415+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:41.393 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:41.400+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:41.707 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:41.715+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 218 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:42.383 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:42.391+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:43.371 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:43.379+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/ceph-osd.7.log: file size changed while zipping 2023-07-29T14:21:44.379 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:44.387+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:44.598 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~265s 2023-07-29T14:21:44.599 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~5s 2023-07-29T14:21:45.414 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:45.422+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:46.395 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:46.403+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:46.708 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:46.716+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 226 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:47.423 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:47.431+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:48.394 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:48.401+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:49.402 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:49.409+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:49.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~270s 2023-07-29T14:21:49.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~10s 2023-07-29T14:21:50.383 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:50.391+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:51.370 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:51.378+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:51.709 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:51.717+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 233 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:52.413 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:52.421+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:53.384 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:53.392+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:54.358 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:54.366+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:55.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~276s 2023-07-29T14:21:55.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~16s 2023-07-29T14:21:55.392 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:55.400+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:56.434 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:56.442+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:56.711 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:21:56.718+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 241 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:21:57.456 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:57.464+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:58.438 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:58.446+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:21:59.463 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:21:59.470+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:00.206 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~281s 2023-07-29T14:22:00.206 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~21s 2023-07-29T14:22:00.496 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:00.504+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:01.485 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/ceph-mon.b.log: file size changed while zipping 2023-07-29T14:22:01.520 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:01.528+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:01.711 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:22:01.720+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 249 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:22:02.543 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:02.551+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:03.521 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:03.529+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:04.484 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:04.491+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:05.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~286s 2023-07-29T14:22:05.409 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~26s 2023-07-29T14:22:05.476 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:05.484+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:06.461 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:06.469+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:06.531 INFO:tasks.ceph:Archiving logs... 2023-07-29T14:22:06.532 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625/remote/smithi040/log 2023-07-29T14:22:06.533 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-29T14:22:06.727 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:22:06.721+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 256 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:22:07.460 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:07.462+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:08.461 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:08.468+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:09.071 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625/remote/smithi045/log 2023-07-29T14:22:09.073 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-29T14:22:09.461 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:09.447+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:10.465 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:10.459+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:10.639 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~291s 2023-07-29T14:22:10.639 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~31s 2023-07-29T14:22:11.085 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1463, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/misc.py", line 1171, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1128, in cluster yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/misc.py", line 1171, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 322, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1197, in cluster raise e File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1179, in cluster remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi045 with status 32: 'sync && sudo umount -f /var/lib/ceph/osd/ceph-7' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1951, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 362, in valgrind_post raise valgrind_exception Exception: saw valgrind issues 2023-07-29T14:22:11.086 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-29T14:22:11.096 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1463, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/misc.py", line 1171, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1128, in cluster yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/misc.py", line 1171, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 322, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1197, in cluster raise e File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1179, in cluster remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi045 with status 32: 'sync && sudo umount -f /var/lib/ceph/osd/ceph-7' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1951, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 362, in valgrind_post raise valgrind_exception Exception: saw valgrind issues 2023-07-29T14:22:11.097 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-29T14:22:11.098 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-29T14:22:11.101 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-29T14:22:11.196 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-29T14:22:11.204 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-29T14:22:11.205 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-29T14:22:11.208 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-29T14:22:11.222 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2023-07-29T14:22:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2023-07-29T14:22:11.226 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-29T14:22:11.226 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2023-07-29T14:22:11.227 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 31 +650us[ +637us] +/- 97ms 2023-07-29T14:22:11.227 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -1011us[-1023us] +/- 60ms 2023-07-29T14:22:11.227 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +932us[ +932us] +/- 85ms 2023-07-29T14:22:11.227 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-29T14:22:11.229 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-29T14:22:11.229 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2023-07-29T14:22:11.229 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 +709us[ +709us] +/- 97ms 2023-07-29T14:22:11.230 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv02.front.sepia.ceph.com 3 6 377 28 -971us[ -932us] +/- 60ms 2023-07-29T14:22:11.230 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +974us[+1013us] +/- 85ms 2023-07-29T14:22:11.230 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-29T14:22:11.230 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-29T14:22:11.239 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-29T14:22:11.239 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-29T14:22:11.247 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-29T14:22:11.376 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-29T14:22:11.453 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:11.461+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:11.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-29T14:22:11.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-29T14:22:11.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-29T14:22:11.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-29T14:22:11.558 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-29T14:22:11.566 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-29T14:22:11.575 INFO:teuthology.task.internal:Duration was 4299.597273 seconds 2023-07-29T14:22:11.576 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-29T14:22:11.584 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-29T14:22:11.585 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-29T14:22:11.587 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-29T14:22:11.637 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-29T14:22:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-29T14:22:11.713 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:22:11.722+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 264 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:22:12.150 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-29T14:22:12.151 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2023-07-29T14:22:12.152 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-29T14:22:12.173 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2023-07-29T14:22:12.174 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-29T14:22:12.200 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-29T14:22:12.201 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-29T14:22:12.216 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-29T14:22:12.274 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-29T14:22:12.275 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-29T14:22:12.289 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-29T14:22:12.474 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:12.481+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:12.920 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-29T14:22:12.931 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-29T14:22:12.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-29T14:22:12.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-29T14:22:12.995 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-29T14:22:13.006 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-29T14:22:13.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-29T14:22:13.036 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2023-07-29T14:22:13.065 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2023-07-29T14:22:13.101 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-29T14:22:13.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-29T14:22:13.116 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-29T14:22:13.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-29T14:22:13.156 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-29T14:22:13.166 INFO:teuthology.task.internal:Transferring archived files... 2023-07-29T14:22:13.167 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625/remote/smithi040 2023-07-29T14:22:13.168 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-29T14:22:13.207 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625/remote/smithi045 2023-07-29T14:22:13.208 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-29T14:22:13.247 INFO:teuthology.task.internal:Removing archive directory... 2023-07-29T14:22:13.248 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-29T14:22:13.250 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-29T14:22:13.297 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-29T14:22:13.313 INFO:teuthology.task.internal:Not uploading archives. 2023-07-29T14:22:13.313 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-29T14:22:13.371 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-29T14:22:13.372 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-29T14:22:13.375 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-29T14:22:13.387 INFO:teuthology.orchestra.run.smithi040.stdout: 263230 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jul 29 14:22 /home/ubuntu/cephtest 2023-07-29T14:22:13.387 INFO:teuthology.orchestra.run.smithi040.stdout: 525435 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 29 13:26 /home/ubuntu/cephtest/ceph.data 2023-07-29T14:22:13.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-29T14:22:13.389 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-07-29T14:22:13.389 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1463, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/misc.py", line 1171, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1128, in cluster yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/misc.py", line 1171, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 322, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1197, in cluster raise e File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1179, in cluster remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi045 with status 32: 'sync && sudo umount -f /var/lib/ceph/osd/ceph-7' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 1951, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_0484097b0572e96f42cf6402cbe0ac8dcb046577/qa/tasks/ceph.py", line 362, in valgrind_post raise valgrind_exception Exception: saw valgrind issues During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_61140164d2579c30ffff3970615e0f82e40c6470/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi040 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-07-29T14:22:13.390 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-29T14:22:13.400 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-29T14:22:13.418 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: saw valgrind issues 2023-07-29T14:22:13.419 INFO:teuthology.nuke:Checking targets against current locks 2023-07-29T14:22:13.423 INFO:teuthology.orchestra.run.smithi045.stdout: 263230 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jul 29 14:22 /home/ubuntu/cephtest 2023-07-29T14:22:13.424 INFO:teuthology.orchestra.run.smithi045.stdout: 395757 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 29 13:26 /home/ubuntu/cephtest/ceph.data 2023-07-29T14:22:13.425 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-07-29T14:22:13.451 DEBUG:teuthology.nuke:shortname: smithi040 2023-07-29T14:22:13.451 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-29T14:22:13.477 DEBUG:teuthology.nuke:shortname: smithi045 2023-07-29T14:22:13.477 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-29T14:22:13.480 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-29 13:02:09.641667', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMq9L1pCZlICkqLKsCQ1cbsTIVQukPVXnoChMPUjBzoDV3riqmHBm32FQWVTGmE7j0ZBfq5L6k7As7l0oYgfcqE='} 2023-07-29T14:22:13.505 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-28_23:11:59-rados-reef-release-distro-default-smithi/7356625', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-29 13:02:09.643293', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRiuvrnGNaq6R3f1uQMQUyTEeSjQdZBYrRH8yP1hs8qaTIr5bJ8hekyG39aJbWdoGjv81vqMy/3vTr8gC0vGJA='} 2023-07-29T14:22:13.513 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:13.521+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:13.541 INFO:teuthology.orchestra.console.smithi040:Power off 2023-07-29T14:22:13.542 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-29T14:22:13.570 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off 2023-07-29T14:22:13.571 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-29T14:22:13.588 INFO:teuthology.orchestra.console.smithi045:Power off 2023-07-29T14:22:13.588 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-29T14:22:13.600 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2023-07-29T14:22:13.611 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off 2023-07-29T14:22:13.612 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-29T14:22:13.634 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2023-07-29T14:22:14.528 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:14.536+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:15.578 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:15.586+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:15.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~296s 2023-07-29T14:22:15.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~37s 2023-07-29T14:22:16.616 INFO:tasks.ceph.osd.7.smithi045.stderr:2023-07-29T14:22:16.624+0000 12a61640 -1 osd.7 1022 heartbeat_check: no reply from 172.21.15.45:6817 osd.6 since back 2023-07-29T14:11:27.428448+0000 front 2023-07-29T14:11:27.429928+0000 (oldest deadline 2023-07-29T14:16:47.621685+0000) 2023-07-29T14:22:16.715 INFO:tasks.ceph.mon.b.smithi045.stderr:2023-07-29T14:22:16.723+0000 d736640 -1 mon.b@1(probing) e1 get_health_metrics reporting 272 slow ops, oldest is osd_failure(failed immediate osd.6 [v2:172.21.15.45:6810/85184,v1:172.21.15.45:6811/85184] for 81sec e1022 v1022) 2023-07-29T14:22:17.601 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-29T14:22:17.635 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-29T14:22:17.712 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2023-07-29T14:22:17.747 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2023-07-29T14:22:20.845 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~302s 2023-07-29T14:22:20.845 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-07-29T14:22:20.845 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~42s 2023-07-29T14:22:20.846 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-07-29T14:22:20.846 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-07-29T14:22:21.715 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-29T14:22:21.748 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-29T14:22:21.826 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2023-07-29T14:22:21.862 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2023-07-29T14:22:21.927 INFO:teuthology.orchestra.console.smithi040:Power off completed 2023-07-29T14:22:25.867 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-29T14:22:25.977 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2023-07-29T14:22:26.078 INFO:teuthology.orchestra.console.smithi045:Power off completed 2023-07-29T14:22:26.182 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} duration: 4299.597272634506 failure_reason: '"2023-07-29T13:39:28.628828+0000 mon.a (mon.0) 1527 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2023-07-29T14:22:26.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-29T14:22:26.270 INFO:teuthology.run:FAIL